Special  Report  62 


U.S.  ARMY  RESEARCH  INSTITUTE 
PROGRAM  IN  BASIC  RESEARCH 
FY  2004 


Research  and  Advanced  Concepts  Office 

(RACO) 


20060317  001 

August  2005 


Approved  for  public  release;  distribution  is  unlimited. 


U.S.  Army  Research  Institute 

for  the  Behavioral  and  Social  Sciences 

A  Directorate  of  the  Department  of  the  Army 
Deputy  Chief  of  Staff,  G1 

Authorized  and  approved  for  distribution: 


ZITA  M.  SIMUTIS 

Director 


NOTICES 

DISTRIBUTION:  Primary  distribution  of  this  Special  Report  has  been  made  by  ARI. 
Please  address  correspondence  concerning  distribution  of  reports  to:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences,  Attn:  DAPE-ARI-MS, 

2511  Jefferson  Davis  Highway,  Arlington,  Virginia  22202-3926. 

FINAL  DISPOSITION:  This  Special  Report  may  be  destroyed  when  it  is  no  longer 
needed.  Please  do  not  return  it  to  the  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences. 

NOTE:  The  findings  in  this  Special  Report  are  not  to  be  construed  as  an  official 
Department  of  the  Army  position,  unless  so  designated  by  other  authorized  documents. 


TABLE  OF  CONTENTS 


The  Basic  Research  Program:  Identifying  New  and  Promising  Technologies .  1 

Research  Objective  1:  Provide  jundamental  knowledge  to  improve 

training  in  complex  environments .  3 

Training  for  Rapid  Interpretation  of  Voluminous  Multi-Modal  Data 

Dennis  Folds .  5-7 

Training  for  Efficient,  Durable,  and  Flexible  Performance  in  the  Military 

Alice  F.  Healy  &  Lyle  E.  Bourne,  Jr .  8-12 

FOCUS:  A  Model  of  Sensemaking 

Klein  &  Associates .  13-17 

Factors  Influencing  the  Design  and  Conduct  of  Effective  Technology-Delivered 
Instruction 

Kurt  Kraiger  &  Annette  Towler .  1 8-22 

Understanding  Aspects  of  Individual  and  Collaborative  Skill  Acquisition  in  Face-to- 
Face  and  Distance  Training  Situations 

Adrienne  Y.  Lee,  Douglas  Gillian,  &  Nancy  Cooke .  23-26 

Exploring  the  Interaction  of  Implicit  and  Explicit  Processes  to  Facilitate  Individual 
Skill  Learning 

Ron  Sun  &  Robert  Mathews .  27-3 1 

The  Integration  of  Implicit  and  Explicit  Knowledge  in  Skill  Acquisition 

Ron  Sun  &  Robert  Mathews .  32-35 

Research  Objective  2:  Provide  jundamental  knowledge  for  improving  leader  and 

team  performance . 37 

The  Leadership  Formula:  P  x  M  x  D 

Reuven  Gal  &  Micha  Popper . 38-40 

Leadership:  Enhancing  Team  Adaptability  in  Dynamic  Settings 

Katherine  J.  Klein,  Steve  W.J.  Kozlowski,  &  YanXiao .  41-45 

Self-Initiated  Self-Development  of  Leadership  Capabilities: 

Toward  Establishing  the  Validity  of  Key  Motivational  Constructs  and 
Assessment  Tools 

Todd  J.  Maurer .  46-49 

Developing  Effective  Military  Leaders:  Facilitating  the  Acquisition  of 
Experience-based,  Tacit  Knowledge 

Robert  J.  Sternberg  &  Cynthia  T.  Matthew . . .  50-53 

Identifying  Individual  Attributes  and  Learning  Principles  that  Foster  Adaptive 
Performance  and  Promote  Rapid  Adaptability  Skill  Acquisition  through  Multiple 
Modes  of  Development 

Stephen  J.  Zaccaro .  54-56 


i 


Research  Objective  3:  Provide  fundamental  knowledge  for  identifying ,  assessing,  and  assigning  quality 
personnel  for  the  evolving  Army .  57 

An  Interactionalist  Analysis  of  Soldier  Retention  Across  Career  Stages  and  Time 

Gilad  Chen  <£  Robert  Ployhart .  58-60 

Measuring  Average  Speed  of  Numerical  Reasoning 

Mark  L  Davison  &  Robert  Semmes .  61-65 

Modeling  the  Strategic  Role  of  Basic  Emotions  with  Cellular  Automata 

Craig  DeLancey .  66-69 

A  Valid,  Culture-Fair  Test  of  Intelligence 

Joseph  Fagan .  70-73 

Testing  a  Theory  of  the  Determinants  of  Individual  Job  Performance  for  United  States 
Army  Junior  Commissioned  Officers 

JeffW.  Johnson .  74-78 

Goal-Driven  Perception  and  Cognition  in  Complex  Social  Environments 

Douglas  T.  Kenrick  &  Steven  L  Neuberg .  79-81 

An  Integration  of  Motivation  Theories 

Avraham  N.  Kluger .  82-87 

International  Military  Education  and  Training:  A  Sociological  Analyses 

Charles  Moskos .  88-89 

Comprehension  and  Memory  of  Spatial  and  Temporal  Event  Components 

Gabriel  A.  Radvansky .  90-95 

Development  and  Empirical  Evaluation  of  a  Theory  of  Socially  Competent  Job 
Performance  for  United  States  Army  Junior  Commissioned  Officers 

Robert  J.  Schneider .  96-100 

Social  Structures  Affecting  Army  Performance 

David  R.  Segal  &  Mady  Wechsler  Segal .  101-105 

Construct  Validating  Aspects  of  the  Theory  of  Successful  Intelligence 
Via  a  Test  Battery  for  Measuring  Mental  Flexibility 

Robert  J.  Sternberg  and  Jens  F.  Beckmann .  1 06- 112 

Temporal  Investigations  into  the  Relationship  Between  Affect  and  Discretionary 
Work  Behavior 

Howard  M.  Weiss  &  Reeshad  S.  Dalai .  113-116 


ii 


THE  BASIC  RESEARCH  PROGRAM: 

Identifying  New  and  Promising  Technologies 


The  Research  and  Advanced  Concepts  Office  (RACO)  research  program  focuses  on  providing 
the  the  basic  research  underpinnings  for  the  personnel,  leader  development,  and  training 
requirements  of  the  future.  This  program  is  a  critical  link  between  the  world  of  behavioral 
science  and  the  military  community.  Searching  out  and  advancing  the  state-of-the-art  methods, 
theories  and  findings  in  behavioral  science,  encouraging  projects  most  likely  to  contribute 
generalizable  scientific  principles  and  new  knowledge,  and  supporting  those  efforts  that  have 
potential  military  relevance  and  likelihood  of  leading  to  applied  behavioral  technology  are 
RACO’s  key  research  goals. 

Universities  conduct  most  of  the  basic  research  program.  RACO  maintains  close  contact  with 
other  basic  behavioral  science  research  organizations  as  well  as  with  ARI’s  applied  researchers 
and  other  relevant  agencies  within  the  Army  and  other  military  services.  These  contacts  help  to 
define  issues  that  require  fundamental  research,  and  facilitate  the  transition  of  basic  research 
results  to  applied  programs  for  eventual  use  by  the  operational  Army. 

In  RACO’s  contract  programs,  a  Broad  Agency  Announcement  (BAA)  is  issued  each  year  to 
solicit  both  concept  papers  and  formal  proposals  relating  to  the  announced  program  research 
areas.  In  a  given  year,  the  BAA  highlights  the  research  objectives  of  special  interest,  and 
provides  an  open  call  for  proposals. 

Onlhe  following  pages,  the  reader  will  find  summaries  of  current  and  recently  completed  RACO 
contracts,  which  began  between  2000  and  early  2005.  There  are  three  current  RACO  program 
objectives,  each  of  which  will  be  discussed  in  detail  below.  These  objectives  are  as  follows: 

1 .  To  provide  fundamental  knowledge  to  improve  training  in  complex  environments, 

2.  To  provide  fundamental  knowledge  to  improve  leader  and  team  performance, 

3.  To  provide  fundamental  knowledge  for  identifying,  assessing,  and  assigning  quality 
personnel  in  the  evolving  Army. 

Training  for  Complex  Environments 

Basic  research  in  this  area  focuses  on  developing  concepts  and  methods  for  training  complex 
tasks  and  for  sustaining  complex  task  performance.  Assessing  the  cognitive  impact  of 
technology  requirements  arising  from  digital,  semi-automated,  and  robotic  systems  on  training 
requirements  is  also  part  of  this  process.  One  of  the  efforts  in  this  area  seeks  to  understand  the 
impact  of  the  voluminous  multi-modal  data  on  performance  and  how  individuals  and  teams 
might  be  trained  to  integrate  and  use  such  rapidly  presented  data.  We  seek  to  identify  unique 
training  principles  and  methods  for  improving  interpersonal  skills  and  team  adaptability  and 
performance.  The  expected  outcome  is  applied  research  testing  the  principles  and  methods 
produced  in  this  work  package  in  Army  training  environments.  The  models  and  theories 
produced  should  be  useful  in  accounting  for  individual  differences  in  training  and  facilitating 
practical,  individualized,  adaptive  training  methods.  These  advances  will  translate  into  improved 
training  methods  in  a  wide  range  of  tasks. 
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Improving  Leader  and  Team  Performance 

Commensurate  with  the  requirements  for  rapidly  developing  adaptable,  flexible  leaders,  the  basic 
research  program  in  improving  leader  performance  is  directed  toward  providing  concepts  and 
methods  for  accelerating  leader  development  and  understanding  and  developing  leader 
adaptability  and  flexibility  in  a  manner  that  can  be  tested  in  the  applied  environment.  One  of  our 
major  efforts  in  this  area  uses  Sternberg’s  theory  of  practical  intelligence  to  develop  new 
techniques  for  acquiring  experience-based,  tacit  knowledge  as  a  means  for  rapidly  developing 
effective  military  leaders.  We  are  also  committed  to  discovering  and  testing  the  basic  cognitive 
principles  that  underlie  effective  leader-team  performance.  Understanding  the  dynamics  of  small 
group  leadership  in  face-to-face  and  distributed  team  environments  is  a  key  aspect  of  the  current 
basic  research  program  in  leadership.  For  example,  one  of  our  research  projects  seeks  to  develop 
leadership  techniques  that  foster  interaction,  communication,  and  trust  in  electronic 
environments.  Another  examines  the  development  of  team  leadership  behaviors  under  the  stress 
of  performing  in  a  metropolitan  hospital  shock  trauma  center  and  in  the  laboratory.  We  anticipate 
that  results  from  this  research  will  make  an  important  contribution  to  understanding  and 
improving  organizational  effectiveness. 

Quality  Personnel  for  the  Evolving  Army 

Identifying  and  measuring  the  aptitudes  and  skills  that  are  unique  to  the  human  performance 
requirements  of  military  service  is  a  major  theme  of  this  basic  research  effort.  As  part  of  this 
process,  we  seek  to  devise  methods  that  assess  mental  flexibility  and  adaptability,  describe  how 
these  attributes  develop,  and  measure  their  contribution  to  performance  and  job  tenure.  For 
example,  one  of  our  efforts  seeks  new  ways  to  conceptualize  and  assess  individual  mental 
flexibility  since  mental  flexibility  plays  a  critical  role  in  the  successful  application  of  the  mental 
processes  needed  for  adapting  to,  selecting,  and  shaping  the  environment  in  novel  situations. 
Exploring  the  sociological  and  psychological  factors  that  could  influence  recruitment,  retention, 
and  performance  is  part  of  the  work  in  this  area  as  well.  Applied  research  that  builds  the 
understanding  of  the  cognitive  processes  entailed  in  flexible  thinking  can  be  used  to  identify 
those  who  are  flexible,  adaptable  thinkers  and  to  develop  these  skills  through  effective 
instructional  programs  that  facilitate  the  use  of  flexible  thinking  strategies  in  a  variety  of 
contexts. 

This  document  provides  a  listing  and  brief  synopsis  of  ongoing  and  recently  completed  research 
efforts.  Project  listings  are  organized  into  the  three  aforementioned  research  objectives.  It  is 
important  to  note,  however,  that  basic  research  is  but  one  of  many  programs  for  which  RACO 
has  responsibility.  Other  programs  in  RACO  include: 

•  Small  Business  Innovative  Research  (SBIR)  Program, 

•  Small  Business  Technology  Transfer  (STTR)  Program, 

•  International  Behavioral  Science  and  Technology  Watch, 

•  Graduate  student  apprenticeship  program  -  Consortium  Research  Fellows  Program  -  with  the 
Consortium  of  Metropolitan  Washington  Universities, 

•  Outreach  efforts  to  Historically  Black  Colleges  and  Universities 
(HBCUs)  and  Minority  Institutions  (Mis),  and 

•  Research  support  in  behavioral  science  for  the  U.S.  Military  Academy. 

Additional  information  about  reports  from  these  research  efforts  is  available  upon  request. 

Paul  A.  Gade,  Chief,  Research  and  Advanced  Concepts  Office 

Laura  Wheeler  Poms,  Consortium  Fellow,  Research  and  Advanced  Concepts  Office 
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RACO  RESEARCH  OBJECTIVE  #1: 


Provide  fundamental  knowledge  to  improve  training  in  complex 
environments. 


Research  under  this  objective  develops  concepts  and  methods  for 
training  complex  tasks  and  sustaining  complex  task  performance.  The 
focus  is  on  understanding  the  cognitive  impact  of  Future  Force 
technology  on  training  requirements,  the  impact  of  voluminous  multi¬ 
modal  data  on  performance,  and  developing  methods  for  improving 
cognitive  skills. 
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Training  For  Rapid  Interpretation  of  Voluminous  Multi-modal  Data 

Contract  #:  D  ASWO 1-02-K-0001  Contract  Dates:  5/1/02- 1/3 1/05 

Institution:  Georgia  Institute  of  Technology  PI:  Dennis  J.  Folds 

Liaison:  Scott  Graham,  IFRU 

Problem(s)/Research  Question(s)  -  How  do  people  make  decisions  when  they  must  process  a 
large  volume  of  data  very  quickly?  What  types  of  errors  do  they  commonly  make?  Can  we  train 
people  to  avoid  common  errors  in  this  type  of  decision  making? 

Technical  Barrier(s)  -  Theoretical  basis  for  understanding  rapid  decision  making  is  not  well 
developed. 

Significance/Impact  for  Basic  Research  -  Understanding  characteristic  errors  in  rapid  decision 
making  will  lead  to  better  understanding  of  underlying  decision  processes. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in 
the  following  Army  applied  research  programs: 

•  Future  Force  Warrior  Training  (WP  215) 

•  FUTURE-TRAIN:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  Combat 
Team  Commanders  and  Staffs  (WP  21 1) 

Overview 

The  primary  goal  of  our  research  is  to  increase  understanding  of  how  individuals  and 
small  teams  make  decisions  when  they  must  process  a  large  amount  of  information  very  rapidly. 
A  second  goal  is  to  determine  how  to  improve  this  ability  through  training.  We  study  the  effects 
of  data  format  (photographs,  audio  files,  video  clips,  text  messages,  computer  graphs,  and  map- 
based  icons)  on  rapid  decision  making.  We  also  study  the  effects  of  overall  information  volume 
(i.e.,  the  total  number  of  information  items  that  a  person  must  process  before  making  a  decision) 
and  information  density  (i.e.,  the  number  of  information  items  relevant  to  the  decision).  We 
identify  the  types  of  errors  that  people  commonly  commit  when  making  decisions  in  this  manner 
and  are  studying  different  ways  to  improve  performance  through  training. 

We  are  conducting  a  series  of  laboratory  experiments  with  individuals  and  small  teams.  The 
specific  objective  for  each  experiment  is  as  follows: 

•  Experiment  1  -  Assess  the  effects  of  data  format,  density,  and  overall  volume  on  decision 
making  performed  by  individuals. 

•  Experiment  2  -  Assess  effectiveness  of  anti-bias  training  for  individuals. 

•  Experiment  3  -  Assess  the  effects  of  data  format,  density,  and  overall  volume  on  decision 
making  performed  by  small  teams. 

•  Experiment  4  -  Assess  effectiveness  of  anti-bias  training  for  small  teams. 

•  Experiment  5  -  Quantify  information  processing  rates  in  rapid  decision  making  for 
individuals. 
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Research  Approach 

Our  research  participants  are  recruited  from  campus  ROTC  units.  They  use  a  simulation 
of  a  hand-held  computer  that  receives  information  over  a  data  link.  The  information  is  in  the 
form  of  a  video  clip,  a  photograph,  a  text  message,  and  audio  file,  a  computer-generated  graph, 
or  an  icon  on  a  map.  These  information  items  arrive  in  rapid  succession  in  the  ‘inbox’  and  must 
be  reviewed  by  the  participant.  They  use  the  information  to  make  decisions  about  potential 
incidents  (such  as  a  car  bomb)  that  they  are  supposed  to  report.  By  design,  the  information  they 
receive  also  contains  a  lot  of  false  leads  that  should  be  rejected.  We  developed  our  patterns  of 
false  leads  so  they  would  tend  to  produce  the  types  of  decision  errors  found  in  previous  research. 
The  equipment  they  use  is  designed  so  that  the  state  it  is  in  tells  us  what  information  the  user  is 
processing,  and  we  can  tell  how  often  a  given  piece  of  information  is  accessed,  and  whether  the 
person  found  it  meaningful. 


Accomplishments 

We  have  completed  the  first  two  major  experiments  and  four  small-scale  pilot  studies. 
In  our  first  experiment,  we  investigated  characteristic  errors  that  occur  as  individuals  make 
decisions  rapidly.  Specifically,  we  studied  six  characteristic  errors  reported  in  other  decision 
making  research  to  determine  whether  those  errors  occur  in  rapid  decision  making,  and  if  so,  to 
what  extent.  In  our  second  experiment  we  assessed  the  potential  effectiveness  of  anti-bias 
training  to  help  reduce  the  occurrence  of  those  characteristic  errors.  Our  first  small-scale  study 
helped  us  refine  the  information  presentation  methods  and  to  set  the  pace  of  information 
presentation  so  that  subjects  would  have  just  enough  time  to  process  all  the  information  and  to 
make  rapid  decisions.  Our  second  small-scale  study  helped  us  develop  the  division  of 
responsibilities  for  a  three-person  team  and  structure  the  tasks  the  team  members  will  perform. 
Our  third  small-scale  study  helped  us  set  the  pace  of  information  presentation  to  teams  and  to 
refine  training  techniques  for  teams.  Our  fourth  small-scale  study  helped  refine  the  conditions 
for  finding  an  upper  limit  for  information  processing  during  rapid  decision  making. 

Our  findings  so  far  show  that  people  can  do  a  fairly  good  job  at  making  these  decisions 
rapidly.  They  are  funding  about  70%  or  more  of  the  incidents  they  should  report  and  are  making 
reports  on  only  about  30%  of  the  false  leads.  The  anti -bias  training  used  in  Experiment  2 
showed  some  advantage,  and  we  learned  some  ways  to  improve  that  training. 


Contributions  to  Basic  Research 

The  current  work  advances  basic  research  in  the  areas  of  naturalistic  decision  making, 
team  decision  making,  and  in  training  to  improve  decision  making.  Previous  research  has 
identified  characteristic  errors  (sometimes  called  biases)  that  humans  commit  when  making 
decisions  based  on  incomplete  or  conflicting  information.  Many  of  these  biases  seem  to  arise 
from  unconscious  strategies  that  reduce  the  effort  required  to  process  information,  thereby 
allowing  quick  decisions  to  be  made.  Our  research  is  examining  whether  these  same  types  (or 
perhaps  different  types)  of  errors  occur  when  an  individual  or  small  team  is  processing  large 
volumes  of  information  very  rapidly.  We  are  also  investigating  whether  the  same  cognitive 
processes  that  produce  the  biases  in  the  first  place  could  be  used  to  help  counter  the  bias  through 
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training.  Specifically,  we  are  studying  whether  individuals  can  learn  to  recognize  the  patterns  of 
information  that  tend  to  produce  errors. 

Potential  Army/Military  Applications 

The  applications  of  this  research  for  the  Army  are  related  to  training  individuals  and 
small  groups  to  assess  information  received  in  the  field  and  to  make  rapid  decisions  based  on  that 
information.  While  in  the  field,  an  Army  Unit  of  Action  may  receive  information  over  a  data 
link  and  download  it  to  a  hand-held  computer,  or  through  normal  voice  communications  over  the 
radio.  Specific  techniques  that  might  be  trained  include  spotting  the  ‘markers’  of  a  situation  that 
tend  to  produce  a  decision  error,  and  the  specific  countermeasures  to  take  once  the  markers  are 
spotted. 


Future  Plans 

During  the  next  year  we  plan  to  complete  all  remaining  experiments.  Experiments  3  and 
5  are  proceeding  in  parallel.  In  Experiment  3,  we  are  investigating  small  teams  (teams  of  three 
individuals)  who  are  working  together  to  rapidly  process  information  and  make  decisions.  In 
Experiment  4,  we  will  study  training  techniques  for  small  teams.  In  Experiment  5,  volunteers 
who  participated  in  Experiment  2  continue  to  perform  rapid  decision  making  tasks  as  we  try  to 
find  an  upper  limit  on  the  rate  of  information  processing  in  this  type  of  task. 
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Training  for  Efficient,  Durable,  and  Flexible 
Performance  in  The  Military 

Contract#:  DASW01-03-K-0002  Contract  Dates:  10/1/02-9/30/06 

Institution:  University  of  Colorado  PI:  Alice  F.  Healy 

Co-PI:  Lyle  E.  Bourne,  Jr. 

Liasons:  Stephen  Goldberg,  SSRU  & 

Robert  Pleban,  IFRU 

Problem(s)/Research  Question(s)  -  This  project  aims  to  test  and  develop  principles  of  training 
that  promote  efficient  learning,  durable  memory,  and  flexible  transfer  performance. 

Technical  Barrier(s)  -  Learning,  memory,  and  transfer  performance  are  not  always  highly 
positively  correlated;  for  example,  rapid  learning  often  leads  to  weak  long-term  retention.  There 
is,  thus,  a  technical  barrier  for  finding  conditions  that  simultaneously  optimize  all  three  aspects 
of  training. 

Significance/Impact  for  Basic  Research  -  The  experiments  conducted  as  a  part  of  this  project 
should  produce  data  that  will  support  the  development  of  a  general  theory  of  training 
encompassing  learning  efficiency,  memory  durability,  and  tranfer  flexibility. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in 
the  following  ARI  applied  programs  (among  others): 

Virtual  Intelligent  Training  for  Objective  Force  Warriors  (WP  294) 

•  FUTURE-TRAIN:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  (WP 
211) 

•  Future  Force  Warrior  Training  (WP  215) 

•  VICTOR:  Virtual  Individual  and  Collective  Training  for  Objective  Force  Warrior  (WP 
233) 

•  Training  for  Interactive  Distributed  Environments  (WP  214) 

•  Simulation-focused  Collective  Aircrew  Training  (WP  231) 

Overview 

On  the  basis  of  earlier  research,  we  were  able  to  develop  a  set  of  training  principles  that 
optimize  the  efficiency  and  durability  of  trained  performance.  But  we  have  also  discovered  that 
conditions  that  lead  to  durability  often  and  perhaps  always,  lead  to  limited  flexibility  or 
adaptivity.  In  fact,  in  our  most  recent  research,  we  have  found  that  training  has  little  or  no 
benefit  if  there  are  discemable  differences  between  the  training  and  testing  situations  in  the 
background  or  context,  even  if  there  are  no  changes  made  in  the  primary  task  requirements.  The 
focus  of  this  project  is,  thus,  to  develop  training  procedures  for  knowledge  and  skill  that  will 
survive  primary  task  or  background  changes  and,  thereby,  produce  flexible,  as  well  as  efficient 
and  durable,  performance  in  military  tasks. 
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Research  Approach 

The  experiments  in  this  project  are  divided  into  three  major  groups.  Experiments  in  the 
first  group  are  designed  to  understand  how  individuals  can  be  trained  to  contend  with  an 
unpredictable  flow  of  information  often  large  in  quantity,  rapidly  presented,  and  ambiguous. 
Experiments  in  the  second  group  are  aimed  to  identify  training  factors  that  promote  adaptive  and 
flexible  performance  in  the  field.  The  final  group  of  experiments  examines  performance  in 
dynamic  and  changing  task  environments.  We  conclude  with  an  effort  to  create,  still  in  the 
laboratory,  a  complex  set  of  tasks  similar  to  those  encountered  by  a  digitally  proficient  pilot 
operating  a  fully  computerized  cockpit,  and  not  unlike  those  of  the  digitally  proficient  “land 
warrior”  soldier  of  tomorrow.  The  major  aim  of  this  set  of  experiments  is  to  determine  the  extent 
to  which  training  principles,  first  established  in  a  simpler  laboratory  task,  generalize  to 
performance  under  these  more  complex. 

Accomplishments 

In  this  summary,  we  focus  on  accomplishments  in  two  of  our  series  of  experiments,  both 
on  information  flow,  which  have  helped  us  to  confirm  previously  proposed  training  principles 
and  to  identify  possible  new  principles. 

The  first  series  of  experiments  dealt  with  processing  large  amounts  of  information. 
Specifically,  three  experiments  examined  mediated  learning  of  facts.  In  each  experiment, 
subjects  learned  facts  during  a  learning  phase  and  were  tested  on  all  facts  during  a  test  phase.  In 
Experiments  1  and  2,  subjects  learned  facts  about  familiar  individuals,  unfamiliar  individuals,  or 
unfamiliar  individuals  associated  with  familiar  individuals  during  an  association  phase.  Using 
prior  knowledge  increased  the  amount  learned  and  recalled  even  when  prior  knowledge  played 
only  a  mediating  role  through  association.  In  Experiment  3,  subjects  learned  facts  about 
unfamiliar  persons  or  unfamiliar  countries,  with  half  the  subjects  in  each  group  first  learning 
associations  between  the  unfamiliar  items  and  familiar  individuals  (mediated)  and  half  not  (low). 
Using  prior  knowledge  to  mediate  learning  aided  fact  recognition  during  learning  and  testing 
(see  Figure  1).  However,  this  advantage  during  learning  was  greater  when  the  new  information 
was  conceptually  similar  to  the  prior  knowledge.  The  results  argue  against  domain  specificity  for 
fact  learning. 
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The  second  series  of  experiments  dealt  with  rapid  and  prolonged  processing  of  information. 
We  published  the  first  two  experiments  in  this  series  (Healy,  Kole,  Buck-Gengler,  &  Bourne, 
2004).  A  new  experiment  was  completed  this  year.  This  experiment  examined  speed-accuracy 
tradeoffs  in  data  entry  and  mental  arithmetic  tasks  in  order  to  test  the  cognitive  antidote 
hypothesis,  according  to  which  adding  cognitive  complications  to  an  otherwise  routine  task  can 
mitigate  the  adverse  effects  of  fatigue  and/or  boredom.  There  were  four  groups  of  subjects 
created  by  crossing  task  (data  entry  or  mental  multiplication)  and  feedback  (bell  for  errors  or  no 
feedback).  In  each  case,  subjects  received  a  stimulus  and  typed  a  four-digit  response.  The  four¬ 
digit  numbers  used  in  the  two  tasks  were  the  same.  The  stimuli  on  each  trial  consisted  of  a  pair 
of  single-digit  multiplication  problems  (e.g.,  3  x  4,  3  x  6)  in  the  case  of  the  mental  multiplication 
task,  but  were  the  numbers  to  be  typed  (e.g.,  1  #  2,  1  #  8)  in  the  case  of  the  data  entry  task.  We 
found  support  for  the  hypothesized  cognitive  antidote  hypothesis.  For  the  data  entry  task,  as 
found  previously,  accuracy  declined  across  session  halves  and  blocks  within  session  halves, 
presumably  because  of  fatigue  and/or  boredom.  In  contrast,  for  the  more  cognitively  challenging 
mental  arithmetic  task,  systematic  improvements  were  found  both  in  speed  and  in  accuracy 
across  session  halves  and  blocks  (see  Figure  2).  However,  the  improvement  in  accuracy  across 
blocks  was  found  only  for  the  mental  arithmetic  conditions  with  feedback.  The  mental 
arithmetic  conditions  with  no  feedback  led  to  stable  performance,  with  no  consistent  change 
across  blocks.  This  experiment  demonstrates  that  adverse  fatigue  effects  on  accuracy  can  be 
offset  by  adding  cognitive  complexities  to  the  task  and/or  by  providing  feedback  on  the 
correctness  of  the  individual  responses.  Accuracy  performance  was  optimized  when  these  two 
antidotes  were  combined. 

Figure  2 
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Contributions  to  Basic  Science 


We  have  proposed  the  following  theoretical  principles  for  optimizing  training:  principle 
of  contextual  interference,  training  difficulty  hypothesis,  depth  of  processing  principle, 
procedural  reinstatement  principle,  specificity  of  training  principle,  functional  task  principle,  and 
strategic-use-of-knowledge  principle.  The  primary  purpose  of  our  experiments  is  to  collect  data 
that  allow  us  to  accept  or  reject  each  of  these  training  principles  that  we  formulated  on  the  basis 
of  our  earlier  basic  laboratory  research  and  to  develop  and  test  new  principles  that  are  supported 
by  these  data.  For  example,  our  recent  study  of  fatigue  allowed  us  to  confirm  a  cognitive 
antidote  hypothesis,  which  could  serve  as  a  new  training  principle. 

Potential  Army/Military  Applications 

Our  experiments  could  be  used  to  drive  applied  research.  To  illustrate  this  potential 
symbiosis  between  basic  and  applied  research,  we  give  two  brief  examples.  First,  our  research 
has  demonstrated  a  high  degree  of  specificity  from  training  to  subsequent  application.  These 
findings  have  crucial  implications  for  military  training  because  instructors  often  assume  that 
teaching  a  primary  task  without  extraneous  secondary  task  requirements  will  benefit  the  learning 
process.  However,  our  findings  imply  that  to  be  effective,  training  must  incorporate  as  many  of 
the  complete  set  of  field  task  requirements  as  possible,  including  all  secondary  task  requirements 
that  might  be  imposed.  In  our  research,  we  are  testing  the  boundary  conditions  of  our  finding 
using  a  variety  of  primary  and  secondary  tasks,  and  we  are  testing  hypotheses  concerning  ways 
to  overcome  this  severe  specificity.  Although  the  tasks  used  in  our  research  are  often 
components  of  military  tasks  and  require  digital  proficiency,  they  are  not  the  real  military  tasks 
currently  being  trained  in  the  Army.  We  hope  that  applied  research  units  are  interested  in  testing 
whether  the  specificity  of  training  principle  we  have  developed  would  apply  to  such  real  tasks 
and  whether  the  methods  we  hope  to  develop  for  overcoming  this  problem  could  be  adapted  to 
improve  military  training.  To  that  end,  we  have  had  discussions  with  Stephen  Goldberg  (Chief, 
Simulator  Systems  Research  Unit)  about  future  collaborative  research  on  the  applicability  of  our 
training  principles  to  simulator  training.  Second,  military  instructors  often  attempt  to  use 
procedures  that  make  learning  easier  and  speed  up  initial  skill  acquisition.  However,  our 
research  demonstrates  that,  although  introducing  sources  of  interference  into  a  task  or  increasing 
the  difficulty  of  the  task  slows  down  initial  skill  acquisition,  these  variations  ultimately  lead  to 
improvements  in  the  durability  and  flexibility  of  the  learned  skill.  In  our  research,  we  are  testing 
this  principle  in  a  variety  of  task  domains,  and  we  are  testing  an  extension  of  this  hypothesis, 
namely  whether  the  difficulty  of  a  secondary  task  has  a  similar  inhibiting  effect  on  initial 
learning  and  facilitating  effect  on  retention  and  transfer  as  does  the  difficulty  of  a  primary  task. 

Future  Plans 

Our  proposal  was  divided  into  three  substantive  sections,  with  each  section  further  divided 
into  three  subsections:  (a)  dealing  with  information  flow,  (b)  factors  promoting  adaptive  and 
flexible  performance,  and  (c)  coping  with  dynamic  environments  and  changing  task  demands. 
We  gave  a  detailed  description  of  at  least  one  major  experiment  in  each  of  these  subsections. 
During  the  first  year  of  the  project,  experiments  were  conducted  on  each  of  the  three  substantive 
sections;  three  major  experiments  were  undertaken  that  year,  one  in  each  of  the  three  sections. 
During  the  second  year  of  the  contract  we  completed  eight  new  experiments.  During  Year  3,  we 
plan  to  conduct  five  new  experiments,  and  we  also  plan  to  write  up  and  submit  for  publication 
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one  or  more  manuscripts  based  on  the  work  completed  in  the  first  two  years  of  the  contract  In 
addition  we  hope  to  extend  our  discussions  with  Dr.  Goldberg  and  initiate  discussions  with  Dr. 
Pleban  about  possible  collaborative  applications  of  our  research  to  military  training.  We  also 
started  to  Program  a  complex  set  of  tasks  simulating  the  battlefield  situation  faced  by  today’s 
so  diers.  When  the  complex  environment  is  fully  programmed,  we  plan  to  design  and  execute 

the  firs  of  several  experiments  examining  the  generalizability  of  previously  established  training 
principles  using  that  simulation.  s 
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FOCUS:  A  Model  of  Sensemaking 

Contract  # :  1435-01-01  -CT -31161  Contract  Dates :  6/ 1  /0 1  -6/ 1  /04 

Institution:  Klein  Associates  Inc.  PI:  Gary  Klein 

Co-PI:  Winston  R.  Sieck 

Problem(s)/Research  Question(s)  -  What  are  the  cognitive  processes  people  follow  in  order  to 
make  sense  of  a  situation?  What  training  interventions  can  be  used  to  improve  individuals’ 
sensemaking  performance? 

Technical  Barrier(s)  -  The  concept  of  sensemaking  (initially  formulated  by  Weick,  1995)  was 
not  developed  from  a  cognitive  perspective.  A  detailed  description  of  sensemaking  mechanisms 
is  necessary  in  order  to  develop  effective  training  and  system  interventions  to  support  this 
cognitive  function. 

Significance/Impact  for  Basic  Research  -  Understanding  the  cognitive  processes  involved  in 
sensemaking  contributes  to  basic  science  by  describing  this  critical  cognitive  process  and 
exploring  ways  to  improve  performance  of  this  process  through  training. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Future  Force  Warrior  Training  (WP  215). 

•  LEADERSHIP:  Developing  leaders  in  a  Changing  Army  (WP  1 03). 

•  VICTOR:  Virtual  Individual  and  Collective  Training  for  Future  Warriors  (WP  233). 
SIMAIR:  Simulation-focused  Collective  Aircrew  Training  (WP  231). 

•  Accelerating  Leader  Development  (WP  269). 

Overview 

Army  leaders  are  often  placed  in  situations  that  are  either  unfamiliar  or  filled  with 
uncertainty.  In  order  to  be  successful,  Army  leaders  must  be  able  to  quickly  and  efficiently  make 
sense  out  of  rapidly  changing  situations.  Without  an  understanding  of  the  situation,  they  cannot 
take  appropriate  actions.  Developing  this  understanding  is  difficult,  especially  when  one  is  faced 
with  conflicting  or  limited  information. 

Sensemaking  is  the  process  by  which  people  develop  their  understanding  in  the  face  of 
surprising  information.  Experts  have  demonstrated  the  ability  to  do  this  with  almost  uncanny 
skill  as  they  put  together  pieces  of  a  puzzle  that  novices  do  not  even  see.  Research  on  the  topic 
has  helped  define  situation  awareness  and  its  different  levels.  However,  we  still  do  not 
understand  the  process  by  which  experts  make  sense  of  situations  and  how  this  sensemaking 
ability  develops  over  time.  Understanding  sensemaking  is  critical  if  the  Army  wants  to  train  and 
develop  this  skill  in  its  future  leaders.  The  problem  is  that  no  one  fully  understands  the  process 
of  sensemaking.  The  purpose  of  this  project  is  to  examine  the  process  of  sensemaking,  how  it 
develops,  and  ways  to  improve  sensemaking  through  training  interventions. 

Research  Approach 

This  study  was  conducted  over  the  course  of  three  years. 
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Year  1  research  focused  on  testing  an  initial  model  of  sensemaking — the  Framework  for 
Observing,  Comprehending,  and  Understanding  Situations  (FOCUS).  Experienced  and  novice 
Information  Operations  (10)  officers  completed  a  series  of  scenarios  designed  to  challenge  the 
participants’  sensemaking.  Participant  comments  during  the  scenarios  were  categorized  and 
coded  based  on  the  types  of  inferences,  speculations,  and  explanations  made.  In  addition, 
researchers  collected  real-life  incidents  of  navigational  sensemaking  in  which  participants 
became  lost  while  driving  and  had  to  reorient  themselves.  Based  on  the  results  from  the 
scenarios  and  incidents,  it  became  evident  that  the  initial  FOCUS  model  did  not  fully  represent 
the  sensemaking  process.  The  research  team  then  developed  the  Data/Frame  Model  of 
Sensemaking  (see  Figure  1). 

In  Year  2,  new  Cognitive  Task  Analysis  data  collection  methods  were  developed  and 
used  to  further  test  the  model.  Cognitive  Task  Analysis  is  a  series  of  methods  and  tools  used  to 
gain  in-depth  access  to  the  mental  processes  that  underlie  performance  of  tasks.  The  researchers 
used  these  methods  during  interviews  with  10  officers  to  elicit  and  explore  incidents  in  which  the 
participants  faced  challenges  to  sensemaking.  Based  on  these  data,  the  Data/Frame  Model  was 
further  refined  and  extended  to  include  six  distinct  sensemaking  functions:  elaborating  the  frame, 
questioning  the  frame,  preserving  the  frame,  comparing  frames,  seeking  a  frame,  and  reframing. 
In  addition,  a  number  of  assertions  regarding  the  nature  of  sensemaking  were  proposed. 

Efforts  in  Year  3  were  devoted  to  accomplishing  two  aims.  First,  tests  of  critical 
assumptions  of  the  model  were  sought.  Second,  a  deeper  understanding  of  expert/novice 
differences  was  sought  in  order  to  further  expand  on  the  training  recommendations  offered  in 
Year  2.  Researchers  recoded  and  reanalyzed  the  10  officer  comments  from  Year  1  to  further 
examine  expert/novice  differences.  Additional  interviews  were  conducted  with  10  officers  at  1st 
IO  Command  in  which  participants  responded  to  a  series  of  scenarios.  These  data  were  coded 
and  analyzed  for  specific  expert/novice  differences. 

Accomplishments 

This  research  effort  has  produced  several  significant  findings  about  the  nature  of 
sensemaking.  First,  we  have  defined  sensemaking  as  the  process  of  fitting  data  into  a  frame,  and 
fitting  a  frame  around  the  data.  People  will  try  to  make  sense  of  data  inputs  they  receive  by 
finding  or  constructing  a  story  to  account  for  the  data.  At  the  same  time,  their  repertoire  of 
stories  will  affect  which  data  elements  they  consider  and  how  they  will  interpret  these  data. 
Thus,  the  frame  and  the  data  work  in  concert  to  generate  an  explanation.  We  also  suggest  that 
data  elements  are  not  perfect  representations  of  the  world,  but  are  constructed.  They  are  sampled 
from  the  available  information  and  defined  in  terms  of  the  available  frames.  They  are 
abstractions  from  the  environment,  and  therefore  distortions  of  reality.  We  assert  that  no  more 
than  three  or  four  of  the  key  data  elements  serve  as  anchors,  which  elicit  the  initial  frame  used  to 
sample  for  more  data  elements. 

In  our  Data/Frame  Model  of  Sensemaking  (Klein  et  al.,  2003),  we  have  differentiated  and 
described  six  activities,  or  building  blocks,  of  sensemaking:  elaborating  the  frame,  questioning 
the  frame,  preserving  the  frame,  comparing  frames,  seeking  a  frame,  and  reframing.  Further,  we 
have  identified  two  sensemaking  cycles  that  encompass  these  activities:  the  assimilation  and 
accommodation  cycles,  corresponding  to  the  usage  suggested  by  Piaget  (1954).  During 


assimilation,  the  person  fits  data  and  information  into  the  frame  or  cognitive  map.  During 
accommodation,  the  person  alters  the  frame  to  fit  the  data. 


signal 

stream 


X 


ELABORATING  THE  FRAME 

seeking  data  jvk  extending  the 

fra  mss*  arlHinn 


DATA 

recognize/ 
construct 
jv  a  frame 


fragmentary 
mental  models 

FRAME  ^  ,  9oals 

define,  connect,) 
and  filter  the  - 
data 


,  inferring  data 


frame;  adding  & 
filling  slots 

a. 

II; 


/f 


.tff1 


RE-FRAMING 

establishing  new  anchors 
recovering  discarded  data 
\  re-interpreting  data 


PRESERVING  THE  FRAME 

knowledge  shields  J4^B4*distortions 


J§!»; 


explaining  away 
data 


...»*sr£. 


«... 


QUESTIONING  THE  FRAME 


inconsistent  date 
anomaly  detection 


* 


violated 
expectancies 


ixation  errors 

COMPARING  FRAMES 

identifying 
alternative  frames 
simultaneous 


sharpening  ^ 
distinctions 


testing 


ms 


SEEKING  A  FRAME 

searching  for  •**. 


information  * 
finding  anchors 

"  * f  ■'  '3Z$ri  "  ’v'^' %kx- 


Figure  1.  Data/Frame  Model  of  Sensemaking. 
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We  also  extended  our  discussion  of  quasi-logical  reasoning  (Klein,  Phillips,  Battaglia, 
Wiggins,  &  Ross,  2002)  into  the  notion  of  “possibility  theory,”  as  opposed  to  probability  theory 
and  deductive  inferences.  The  reasoning  we  observed  in  our  incident  data  rarely  employed 
deductive  logic.  Instead,  it  was  aimed  at  employing  inferences  in  order  to  identify  anchors  that 
could  be  used  in  sensemaking.  We  assert  that  quasi-logical  reasoning  and  possibility  theory  are 
strengths  in  the  process  of  sensemaking. 


Consistent  with  prior  research  (Barrows,  Feightner,  Neufeld,  &  Norman,  1978;  Chase  & 
Simon,  1973;  Elstein,  1989),  our  findings  also  suggest  that  expert/novice  differences  in 
sensemaking  performance  are  not  due  to  superior  reasoning  and  inferencing  strategies  on  the 
part  of  the  expert,  but  rather  to  the  quality  of  the  frame  that  is  brought  to  bear.  Experts  have 
more  factual  knowledge  about  their  domain,  they  have  built  up  more  experiences,  and  they  have 
more  knowledge  as  to  cause  and  effect  relationships.  They  are  more  likely  to  generate  an 
accurate  explanation  of  the  situation  than  novices  because  their  frame  enables  them  to  select  the 
right  data  from  the  environment,  interpret  them  more  accurately,  and  see  more  pertinent  patterns 
and  connections  in  the  data  stream. 


Our  research  also  suggests  that  developing  a  comprehensive  mental  model  for  a  complex, 
open  system  is  unrealistic.  Instead  most  people,  and  even  most  experts,  rely  on  fragments  of 
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local  cause-effect  connections,  rules  of  thumb,  patterns  of  cues,  and  other  linkages  and 
relationships  between  cues  and  information  to  guide  the  sensemaking  process  (and  indeed  other 
high-level  cognitive  processes).  We  believe  that  a  set  of  fragmentary  mental  models  contribute 
to  the  frame  that  is  constructed  by  the  sensemaker,  therefore  guiding  the  selection  and 
inteipretation  of  data. 

We  have  elaborated  the  model  such  that  it  can  be  used  to  generate  testable  hypotheses — 
including  non-trivial  and  non-obvious  hypotheses — and  have  presented  several  hypotheses  as 
starting  points  for  model  testing. 

We  found  that  10  experts  exhibit  a  stronger  delineation  between  sensemaking  and 
decision-making  processes.  Experts  attempt  to  first  clearly  understand  the  situation  they  face, 
and  then  proceed  to  formulate  courses  of  action  and  other  decision-making  activities. 
Furthermore,  10  experts  are  much  more  likely  than  others  to  formulate  high-quality  questions 
geared  towards  understanding  anomalous  evidence.  The  kinds  of  questions  10  experts  asked 
varied  in  terms  of  generality. 

Contributions  to  Basic  Research 

The  sensemaking  model  developed  in  this  effort  describes  a  higher-order  cognitive 
process  that  is  a  critical  component  of  decision-making  success  in  naturalistic  environments. 
Sensemaking  drives  the  ability  to  comprehend  the  meaning  and  relevance  of  large  amounts  of 
ambiguous  data.  This  ability  separates  the  individual  who  is  on  top  of  the  situation,  and  able  to 
form  expectancies  to  guide  smooth  performance,  from  the  individual  who  is  continually 
surprised  and  confused  and  acts  hesitantly  in  responding  to  the  crisis  of  the  moment.  This  effort 
contributes  to  basic  science  by  describing  this  critical  cognitive  process  and  exploring  ways  to 
improve  performance  of  this  process  through  training. 

Potential  Army/Military  Applications 

An  understanding  of  sensemaking  is  relevant  to  Army  efforts  to  design  various  types  of 
decision  support  systems.  By  describing  how  humans  actually  use  information  to  develop  and 
revise  interpretations  of  a  situation,  we  can  develop  command  and  control  and  other  technologies 
that  work  in  concert  with  the  human  and  support  his/her  natural  process  for  filtering  information 
and  building  and  maintaining  situation  awareness.  Our  findings-to-date  on  the  nature  of  the 
sensemaking  process  offer  initial  guidance  on  several  important  issues  in  the  field  of  decision 
support,  such  as  the  effect  of  information  rate  on  skilled  performance  and  possible  evaluation 
metrics  for  systems  intended  to  bolster  sensemaking  activities. 

Defining  the  process  of  sensemaking  is  also  relevant  to  Army  training  needs.  Our 
findings  illustrate  areas  in  which  sensemaking  can  break  down  and  even  fail.  Training  programs 
can  be  structured  to  enable  practice  and  feedback  in  key  sensemaking  activities.  Our  findings 
also  highlight  areas  of  expert/novice  differences.  Training  scenarios  can  be  structured  to 
deliberately  build  sensemaking  expertise  in  non-experts  through  the  development  of  mental 
models,  fragmentary  mental  models,  and  anticipatory  thinking  skills. 
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Final  Summary 

This  research  effort  explored  sensemaking — the  process  people  use  to  make  sense  of 
situations.  This  research  included  model  development,  testing,  and  implications  for  training  that 
can  help  Army  personnel  to  build  and  recover  their  understanding  of  situations  more  efficiently. 

This  project  resulted  in  the  development  of  the  Data/Frame  Model  of  Sensemaking, 
which  details  critical  features  of  sensemaking  processes.  This  Data/Frame  Model  proposes  that 
data  and  frames  (theories)  result  in  understanding  via  a  highly  interactive  process.  Primary 
sensemaking  activities  include  frame  elaboration,  questioning,  comparison,  preservation, 
seeking,  as  well  as  reframing.  Early  research  efforts  developed  the  Data/Frame  Model  and 
identified  the  components  and  linkages  that  describe  how  individuals  engage  in  sensemaking. 
Later  efforts  focused  on  expert  and  novice  differences  in  frame  questioning,  and  issues  regarding 
whether  expert  questioning  strategies  could  be  identified  and  taught. 

The  ability  of  commanders  at  all  levels  to  quickly  and  efficiently  come  to  understand 
situations  is  of  critical  importance  to  the  success  of  modem  military  missions.  The  results  of  this 
research  effort  are  currently  being  applied  in  several  areas.  First,  the  Data/Frame  Model  is 
informing  the  development  of  information  technology  that  aids  sensemaking.  The  focus  of  a 
Phase  I  project  has  been  to  provide  military  system  designers  with  a  methodology  and  support 
tools  for  capturing  and  leveraging  sensemaking  requirements.  Currently,  these  support  tools  do 
not  exist.  The  Phase  I  effort  successfully  developed  and  piloted  a  sensemaking  methodology  that 
is  rooted  in  the  Data/Frame  Model.  Second,  the  model  and  empirical  results  are  also  proving 
useful  to  the  development  of  cognitive  metrics  for  intelligence  analysis  and  related  domains. 
Finally,  the  insights  that  have  been  gleaned  from  the  initial  basic  research  into  expert  and  novice 
differences  in  sensemaking  strategies  should  prove  useful  towards  the  development  of  training 
methods  and  materials  for  promoting  sensemaking  skills  across  domains. 
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Factors  Influencing  the  Design  and  Conduct 
Of  Effective  Technology-Delivered  Instruction 

Contract  #:  DASW01-04-K-0002  Contract  Dates:  03/01/04-02/28/06 

Institution:  The  University  of  Tulsa  PI:  Kurt  Kraiger 

Illinois  Institute  of  Technology  Co-PI:  Annette  Towler  (IIT) 

Liaisons:  Scott  Graham,  IFRU; 

Jim  Belanich,  RACO 

Problem(s)/Research  Question(s)  -  Do  seductive  details  (i.e.,  entertaining  and  interesting 
information  that  is  only  tangential  to  the  main  theme  of  the  topic)  and  on-screen  text  hinder 
learning  of  declarative  and  procedural  knowledge  in  a  computer-based  training  environment? 
Given  the  seductive  details  effect,  can  good  instructional  design  principles  reverse  the  effect? 
Does  the  impact  of  seductive  details  on  learning  depend  on  trainees’  learning  styles? 

Technical  Barrier(s)  -  In  order  to  test  instructional  design  principles,  it  is  first  necessary  to 
establish  a  seductive  details  effect  for  training  on  real-world  tasks.  Previous  research  has 
focused  exclusively  on  declarative  knowledge  and  the  majority  of  the  studies  have  been 
conducted  by  a  single  researcher,  Richard  Mayer,  and  his  colleagues.  In  addition,  previous 
research  has  not  examined  whether  trainees’  learning  styles  affects  the  extent  to  which  seductive 
details  impair  learning. 

Significance/Impact  for  Basic  Research  -  The  results  will  allow  us  to  assess  the  extent  to 
which  including  seductive  details  in  training  impairs  both  trainees’  ability  to  recall  the  facts  and 
principles  taught  in  training  and  their  ability  to  perform  the  skills  that  were  taught.  In  the  long¬ 
term,  this  research  will  enable  the  military  to  design  more  effective  computer-based  training. 

Potential  Transitions  -  The  knowledge  acquired  from  this  basic  research  can  be  used  to  test  a 
variety  of  instructional  design  principles.  In  the  past,  most  design  principles  have  been  tested 
only  in  comparison  to  a  “no-treatment”  or  “placebo”  effect.  Once  we  establish  the  seductive 
details  effect,  we  can  test  the  effectiveness  on  a  variety  of  instructional  design  principles  by 
examining  their  ability  to  “undo”  the  seductive  detail  effect. 

Overview 

One  common  strategy  for  enhancing  any  type  of  training  is  to  make  the  content  as 
interesting  as  possible.  Intuitively,  these  strategies  should  increase  intrinsic  motivation  in 
learners,  resulting  in  greater  appreciation  for  the  material  and  greater  persistence  in  learning 
environments.  Obviously,  interesting  information  that  is  central  to  the  main  topic  is  useful  and 
can  facilitate  trainees’  learning.  However,  trainers  or  instructional  designers  tend  to  include 
interesting  information  that  is  unrelated  to  the  central  theme  to  spice  up  mundane  information. 
This  phenomenon  is  called  the  “seductive  details  effect”  and  refers  to  essentially  “highly 
interesting  and  entertaining  information  that  is  only  tangentially  related  to  the  topic  but  is 
irrelevant  to  the  author’s  intended  theme”  (Harp  &  Mayer,  1998,  p.  1).  Instructional  material  can 
be  seductive  through  inclusion  of  illustrations  or  text  that  is  interesting  and  entertaining  but 
tangential  to  the  topic  (e.g.,  adding  a  picture  of  a  professional  football  team  to  a  training  module 
on  teamwork). 
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Although  one  could  argue  that  the  inclusion  of  seductive  details  increases  trainees’ 
emotional  interest  in  the  topic,  there  is  evidence  suggesting  that  the  inclusion  of  this  material 
disrupts  trainees’  concentration  on  the  main  themes  of  the  material  (Haip  &  Mayer,  1998; 
Mayer,  Heiser,  &  Lonn,  2001).  Thus,  trainees  learn  less  in  courses  where  seductive  details  are 
included  along  with  the  presentation  of  material  relevant  to  the  topic  of  the  course. 

However,  previous  research  on  the  seductive  details  effect  has  focused  exclusively  on 
declarative  knowledge  and  has  primarily  been  conducted  by  a  single  researcher,  Richard  Mayer. 
The  goal  of  the  initial  study  is  to  assess  the  impact  of  seductive  details  and  on-screen  text  on 
trainee’s  ability  to  perform  the  skills  taught  in  a  computer-based  training  course.  This  will  allow 
us  to  assess  the  impact  of  seductive  details  on  procedural  knowledge.  Based  on  previous 
research,  it  was  hypothesized  that  trainees  would  learn  less  when  seductive  details  were  included 
in  training  than  when  training  did  not  include  seductive  details. 

Research  Approach 

Undergraduates  participate  in  a  10  minute  computer-based  course  on  how  to  use  Excel. 
The  course  demonstrates  several  functions  in  Excel  that  could  be  used  to  select  a  city  to  live  in 
based  on  the  city’s  average  rainfall  and  temperature.  Participants  are  randomly  assigned  to  one 
of  four  experimental  conditions:  1)  training  includes  both  seductive  details  and  on-screen  text,  2) 
training  includes  seductive  details  but  not  on-screen  text,  3)  training  includes  on-screen  text  but 
not  seductive  details,  4)  training  did  not  include  seductive  details  or  on-screen  text.  On-screen 
text  summarizes  key  points  covered  in  training,  while  seductive  details  included  interesting  facts 
about  the  data  used  in  training  such  as  “Tree  crickets  are  called  the  poor  man's  thermometer 
because  temperature  directly  affects  their  rate  of  activity.  Listen  for  a  cricket  and  count  the 
number  of  chirps  it  makes  in  fifteen  seconds.  Add  37.  The  sum  will  be  the  Fahrenheit 
temperature  (almost  exactly!).” 

After  viewing  the  training  video,  trainees  complete  a  multiple  choice  test  on  their  ability 
to  recall  the  facts  taught  in  training  and  are  asked  to  perform  the  skills  that  were  demonstrated  in 
the  video.  Trainees  then  fill  out  several  measures  for  half  an  hour  to  distract  them  from  thinking 
about  the  training  course  including  an  assessment  of  four  learning  styles:  auditory  (i.e.,  learning 
from  listening  and  discussing  the  material),  visual  (i.e.,  learning  from  demonstration  and  visual 
aids),  tactile  (i.e.,  learning  from  performing  the  task),  and  reading  (i.e.,  learning  from  reading  and 
writing).  Finally,  trainees  are  given  one  more  test  to  assess  their  ability  to  perform  the  tasks 
taught  in  training.  This  performance  measure  is  used  to  assess  transfer. 

Accomplishments 

Currently,  data  from  45  participants  have  been  collected  from  undergraduates  at  the 
University  of  Tulsa.  Data  from  at  least  40  participants  at  the  Illinois  Institute  of  technology  will 
be  collected  by  December  2004.  The  results  below  reflect  the  analyses  performed  on  the  first  45 
participants.  The  two  datasets  will  be  merged  in  December  2004  to  assess  the  effect  of  seductive 
details  on  learning. 

Seductive  details  did  not  have  a  large  impact  on  trainees’  ability  to  perform  the  skills 
taught  in  training  immediately  after  viewing  the  video  or  on  the  transfer  task.  The  average  level 
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of  performance  immediately  following  training  and  on  the  transfer  task  was  slightly  better  for  the 
seductive  details  group  than  the  no  seductive  details  group  (See  Table  1).  Thus,  scores  on  the 
performance  tasks  failed  to  support  the  hypothesis.  However,  seductive  details  did  affect 
trainees’  scores  on  the  multiple  choice  test  in  the  hypothesized  direction.  The  average  level  of 
performance  for  the  seductive  details  group  was  slightly  worse  than  the  average  level  of 
performance  of  the  no  seductive  details  group. 

Table  1.  Means  and  standard  deviations  for  scores  on  the  three  posttests. 


Performance  task  Multiple  choice 

following  training  Transfer  task  test 


Mean 

SD 

Mean 

SD 

Mean 

SD 

Seductive  details 

No 

.66 

.20 

.65 

.18 

.72 

.15 

Yes 

.68 

.24 

.71 

.20 

.66 

.20 

On-screen  text 

No 

.63 

.24 

.65 

.18 

.67 

.19 

Yes 

.71 

.21 

.72 

.20 

.71 

.18 

On-screen  text  improved  trainees’  ability  to  perform  the  skills  taught  in  training 
immediately  after  viewing  the  video  and  on  the  transfer  task  and  increased  their  scores  on  the 
multiple  choice  exam  (See  Table  1). 

Seductive  details  interacted  with  trainees  previous  experience  with  Excel  when  predicting 
scores  on  the  performance  test  following  training  (i.e.,  the  impact  of  seductive  details  on  learning 
depends  on  trainees’  familiarity  with  Excel)  (See  Figure  1).  In  the  no  seductive  details 
condition,  trainees  who  were  unfamiliar  with  Excel  outperformed  trainees  who  were  familiar 
with  Excel.  In  the  seductive  details  condition,  trainees  who  were  familiar  with  Excel  performed 
slightly  better  than  trainees  who  were  unfamiliar  with  Excel. 

Seductive  details  interacted  with  trainees’  auditory  learning  style  when  predicting  scores 
on  the  performance  test  following  training  (See  Figure  2).  Trainees  in  the  seductive  details 
condition  performed  poorly  on  the  performance  test  following  training,  regardless  of  whether 
they  had  an  auditory  learning  style.  In  the  no  seductive  details,  trainees  with  an  auditory  learning 
style  outperformed  those  without  an  auditory  learning  style. 
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Figure  1 


Figure  2 


Seductive  details  interacted  with  trainees’  tactile  learning  style  when  predicting  scores  on  the 
multiple  choice  and  performance  tests  following  training  (See  Figures  3  &  4).  Trainees  in  the 
seductive  details  condition  performed  poorly  on  the  multiple  choice  and  performance  tests 
following  training,  regardless  of  whether  they  had  a  tactile  learning  style.  In  the  no  seductive 
details,  trainees  with  a  tactile  learning  style  outperformed  those  without  a  tactile  learning  style. 


Figure  3  Figure  4 


Contributions  to  Basic  Science 

Results  to  date  suggest  that  seductive  detail  effects  are  not  as  straightforward  as 
previously  thought.  For  example,  our  results  show  that  seductive  details  have  a  greater  effect  on 
declarative  knowledge  than  procedural  knowledge,  and  that  the  impact  of  seductive  details  may 
be  moderated  by  trainees’  learning  styles.  More  specifically,  the  results  indicate:  1)  seductive 
details  did  not  have  a  large  impact  on  trainees’  ability  to  perform  the  skills  taught  in  training  2) 
seductive  details  did  impair  performance  on  a  multiple  choice  test  on  the  same  material  3)  the 
impact  of  seductive  details  on  learning  outcomes  depends  on  trainees’  learning  styles. 

However,  the  current  results  should  be  interpreted  in  conjunction  with  previous  research.  Mayer 
and  others  demonstrated  seductive  details  impair  trainees’  ability  to  recall  the  information  taught 
in  training  and  to  produce  creative  solutions  to  a  transfer  task  in  training  on  lightening  formation 
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(Mayer,  Heiser,  &  Lonn,  2001;  Harp  &  Mayer.  1997;  1998).  The  discrepancy  between  Mayer’s 
findings  and  the  current  results  may  indicate:  1)  seductive  details  impair  the  ability  to  recall 
declarative  knowledge  but  not  procedural  knowledge  2)  the  ability  of  the  researcher  to  detect  the 
seductive  details  effect  depends  on  the  test  format  3)  there  are  differences  across  researchers  in 
their  ability  to  detect  the  seductive  details  effect. 

Potential  Army/Military  Applications 

The  initial  study  is  intended  to  create  a  baseline  effect  for  later  testing  of  instructional 
design  principles,  and  it  is  these  later  studies  that  will  have  the  greatest  application  to  military 
training  practices.  However,  or  partial  replication  of  the  seductive  detail  effect  in  a  computer- 
based  training  environment  suggests  that  instructional  designers  and  trainers  in  the  military 
should  be  cautious  about  including  in  training  content  too  many  interesting  examples  that  are 
irrelevant  to  the  main  points,  particularly  when  the  training  focuses  on  the  acquisition  of 
declarative  knowledge. 

Future  Plans 

The  goal  of  the  next  several  research  experiments  will  be  to: 

1.  Test  whether  discrepancies  between  Mayer’s  findings  and  the  current  results  are  due  to 
the  test  format  or  type  of  knowledge  taught  in  training  (i.e.,  declarative  or  procedural).  A 
computer-based  Excel  course  will  be  developed  to  teach  several  skills  (i.e.,  procedural 
knowledge)  and  several  facts  about  Excel  (i.e.,  declarative  knowledge).  Trainees  will 
then  complete  a  multiple  choice  and  performance  test  on  both  the  declarative  knowledge 
and  procedural  knowledge  components  of  training. 

2.  Replicate  Mayer’s  research  (Harp  &  Mayer,  1997).  Students  will  be  given  a  booklet  to 
review  on  lightening  formation  to  assess  the  impact  of  seductive  details  on  learning 
outcomes.  This  will  allow  us  to  assess  whether  the  discrepancy  between  Mayer’s  results 
and  the  current  findings  are  due  to  differences  across  researchers. 

3.  Test  whether  seductive  details  have  a  deleterious  effect  depending  on  whether  they  are 
dependent  on  instructional  content  or  not.  This  study  originates  from  Scraw’s  belief  that 
seductive  details  represent  a  heterogeneous  class  of  information  with  participants 
devoting  more  processing  time  to  context-dependent  seductive  details  than  to  context- 
independent  seductive  details.  We  plan  to  replicate  Schraw’s  study  using  a  skills-based 
task  -  learning  how  to  perform  Mailmerge  in  Microsoft  Word. 

4.  Design  a  training  intervention  around  the  use  of  advanced  organizers  to  reduce  the 
seductive  details  effect. 

5.  Validate  a  measure  of  learning  styles  to  be  used  as  a  covariate  in  future  studies. 
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Understanding  Aspects  of  Individual  and  Collaborative  Skill  Acquisition  in 
Face-To-Face  and  Distance  Training  Situations 

Contract#:  DASW01-01-K0003  Contract  Date:  9/1/01-8/31/04 

Institution:  New  Mexico  State  University  PI:  Adrienne  Y.  Lee 

Co-PIs:  Douglas  Gillan  &  Nancy  Cooke 

Problem(s)/Research  Question(s)  -  How  do  teams  learn  at  a  distance?  How  does  changing  the 
context  between  training  and  test  affect  performance?  What  kind  of  training  can  we  provide  to 
improve  that  transfer  between  contexts? 

Technical  Barrier(s)  -  Cognitive  issues  for  team/group  training  is  a  relatively  recent  topic  of 
study.  Teams  of  more  than  two  individuals  are  difficult  to  acquire  and  sustain  for  multi-day 
experiments.  Improved  technology  allows  for  both  more  sophisticated  studies  and  better 
measurement  tools;  however,  this  also  means  relying  on  outside  vendors  (to  supply  the  advance 
software  &  programming  support)  and  also  on-site  technical  support. 

Significance/Impact  for  Basic  Research  -  Through  this  research,  we  will  have  a  better 
understanding  of  the  factors  that  affect  team  learning  across  contexts  (i.e.,  what  affect  interacting 
face-to-face  versus  communicating  via  technology,  what  affect  the  context  of  complex  software 
plays  on  this  interaction). 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 
•Training  for  Interactive  Distributed  Environments  (WP  214). 

•Other  training  programs. 

Overview 

The  purpose  of  this  research  is  to  examine:  a)  whether  teams  training  together  in  the 
same  location  (co-located)  will  train  more  quickly  and  efficiently  than  teams  who  are  not  in  the 
same  location  (distributed);  b)  whether  testing  in  the  same  situation/context  (co-located  to  co¬ 
located  and  distributed  to  distributed)  is  better  than  training  in  different  situations/contexts  (co¬ 
located  to  distributed  and  distributed  to  co-located);  c)  whether  we  can  develop  training  to 
improve  the  transfer  between  contexts;  and  d)  whether  the  training  will  improve  learning. 

Research  Approach 

This  research  asked  teams  of  three  individuals  to  learn  a  complex  task  and  then  use  that 
knowledge  to  work  together  to  solve  some  new  tasks.  Originally,  for  the  tasks,  individuals  were 
asked  to  learn  how  to  operate  an  Unmanned  Air  Vehicle  (UAV)  where  the  three  individuals, 
each  with  different  jobs,  must  fly  a  UAV  to  various  locations  and  take  pictures  of  specific 
targets.  After  an  initial  study,  the  new  focus  shifted  to  peacekeeping  missions  where  three 
individuals  manage  different  resources  in  Support  and  Stability  Operations  (SASO).  Both  of 
these  contexts  are  simulated  in  the  laboratory  and  training/testing  is  performed  using  the 
following  methodology.  First  individuals  learn  the  complex  task  either  together  (face-to-face, 
co-located)  or  apart  (distance,  distributed).  Then  a  new,  equally  complex  task  is  chosen  and  the 
individuals  are  asked  to  perform  the  new  task.  They  are  either  asked  to  stay  in  the  same  context 
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or  change  to  the  other  context.  Initially,  for  the  UAV  task,  co-located  context,  the  computer 
consoles  were  in  a  semi-circle  that  allowed  individuals  to  speak  over-the-shoulder  to  each  other; 
for  the  distance  context,  the  individuals  were  separated  by  a  screen,  with  one  in  another  room. 
For  the  SASO  task,  co-located,  individuals  are  facing  computes  in  a  circular  configuration  which 
allows  them  to  talk  to  each  other  easily;  for  the  distance  context,  the  individuals  are  in 
completely  separate  rooms  with  sound  material  on  the  walls.  The  new  technology  allows  for 
smaller,  lighter  computer  equipment  that  can  be  easily  moved  to  different  rooms  and 
communication  occurs  through  light  headsets. 

Accomplishments 

In  the  Spring  and  Fall  2002,  the  first  research  on  co-located/distance  team  training  and 
transfer  was  performed  in  the  UAV  CERTT  lab.  Performance  results  are  shown  below. 
Although  all  teams  received  the  same  initial  (declarative)  knowledge  training,  distance  teams 
outperformed  co-located  teams  throughout  the  initial  hands-on  training  sessions  and  after 
transfer.  The  distance  teams  reached  a  plateau  but  this  was  probably  due  to  a  lack  of  feedback. 
After  transfer,  all  teams  experienced  a  decrement  in  performance  (similar  to  Singley  & 
Anderson,  1989)  but  the  distance  teams  (i.e.,  those  who  trained  at  a  distance  and  either  stayed  in 
the  same  distance  condition  or  transferred  to  a  co-located  condition)  showed  a  greater 
improvement  for  the  two  training  missions  after  transfer.  These  results  may  have  been  due  to 
the  small  numbers  of  teams  or  the  experimental  setup  in  the  UAV  lab  which  may  have 
suppressed  co-located  performance. 


Figure  1 .  Results  from  Experiment  1. 

Note  change  in  context  occurs  at  Mission  7.  CC=  co-located;  DD=  distributed;  CD  =  co-located 
to  distributed;  DC  =  distributed  to  co-located. 

In  2003  we  developed  the  SASO  lab  and  tested  the  software  both  among  the  graduate 
student  assistants  and  with  practice  teams.  We  were  able  to  get  necessary  changes  to  the  SASO 
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software  and  to  debug  that  software,  install  video  and  audio  recording,  re-configure  a  lab  space 
to  house  the  hardware  and  train  assistants  on  the  task.  From  Spring  2004  and  continuing  through 
this  fall,  we  have  been  replicating  the  study  in  the  SASO  lab.  The  main  differences  in  physical 
context  were  described  above;  however,  we  added  a  communication  training  component.  We 
have  run  50  teams  so  far  and  have  a  goal  for  this  first  experiment  of  60  teams.  (We  increased  the 
number  in  order  to  test  the  technology  of  push-to-talk  versus  no  push-to-talk.)  Preliminary 
results  for  some  of  the  teams  are  shown  in  Figure  2.  So  far  the  results  using  only  the  no  push-to- 
talk  data  follow  the  same  pattern  as  the  UAV  study  but  do  not  show  as  clear  a  distance  training 
advantage.  In  the  spring,  we  intend  to  replicate  the  study  again  in  the  SASO  lab  using  a  change 
in  context  in  the  actual  SASO  software  (described  below).  In  this  way,  we  can  see  whether  the 
change  in  the  technology  has  a  bigger  effect  than  a  change  in  the  physical  context. 
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Figure  2.  Results  to  date  (9/1/2004).  Only  34  teams  included. 

Contributions  to  Basic  Research 

In  recent  years,  the  advancement  of  military  technology  has  resulted  in  highly  complex 
skills  that  need  to  be  acquired  and  maintained  to  use  that  technology  (Barry  &  Runyan,  1995). 
Consequently,  the  demand  for  ongoing  innovations  including  distance  training  has  increased. 
Distance  training  has  been  show  to  be  extremely  cost  effective  and  has  produced  similar  learning 
outcomes.  Although  extensive  research  on  distance  learning  exists,  few  studies  have  focused  on 
group  (collaborative)  distance  training  delivered  by  intemet/web-based  technologies. 

A  parallel  development  has  occurred  in  the  area  of  team  training.  Even  though  group 
(collaborative)  learning  has  been  the  focus  of  many  studies  (Slavin,  1 996)  and  progress  has  been 
made  on  knowledge  measurement  at  the  individual  level,  the  measurement  of  team  knowledge, 
and  team  cognition  in  general,  is  still  in  its  infancy  (Cooke,  Salas,  Cannon-Bowers  &  Stout, 
2000).  Thus,  research  on  how  to  improve  team  distance  learning  can  contribute  significantly  to 
the  field  in  several  areas  including  distance  education  and  across  context  team  transfer. 
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Interest  in  distance  education  has  increased  due  to  a  concerted  effort  on  the  part  of  public 
educational  institutions  to  reach  new  populations  of  students  who  might  not  otherwise  be  able  to 
attend.  With  advancements  in  technology,  this  has  been  possible  (particularly  in  the  use  of  web- 
based  training).  On  the  other  hand,  few  controlled  studies  exist  to  determine  what  is  truly 
beneficial. 

Controlled  studies  for  team  distance  training  could  also  contribute  to  the  transfer  literature 
generally.  Research  has  been  performed  on  individual  transfer  (Singley  &  Anderson).  Although 
specifications  for  transfer  are  delineated,  research  on  context  effects  has  not  been  as  conclusive  (see 
McDaniel,  Anderson,  Einstein  &  O’Halloran,  1989  or  Wickens,  1987).  The  current  research  project 
is  focused  upon  describing  the  effects  of  a  particular  physical  context  change  on  team  performance 
(with  a  focus  on  cognitive  skills  but  also  some  measurement  of  social  skills). 

Potential  Army/Military  Applications 

The  applications  to  the  military  are  expected  to  be  in  principles  for  distance  training  and 
transfer  (changing  from  one  situation  to  another)  situations.  Specifically,  these  studies  are 
designed  to  determine  what  conditions  promote  distance  training  and  transfer  and  what  training 
is  needed  to  improve  team  and  distance  transfer.  In  addition,  within  the  studies,  various 
assessment  measures  will  be  used  that  will  focus  on  individual  and  team  cognitive  and  social 
(social-cultural)  skill  development. 

Current  Research  and  Future  Plans 

Currently,  we  are  running  teams  to  complete  the  first  study.  For  the  second  study,  we 
would  like  to  identify  places  where  training  could  be  improved  and  provide  that  additional 
training.  In  addition,  we  will  examine  a  context  change  in  software.  Specifically,  the  current 
SASO  software  focuses  on  Bosnia.  We  would  like  to  switch  the  surface  features  of  the  Bosnia 
software  to  reflect  Darfur.  A  graduate  student  research  assistant  is  researching  Darfur  and  we 
will  be  developing  a  whole  suite  of  training  software  for  that  region.  In  the  second  study, 
participants  will  train  on  either  the  Bosnia  or  Darfur  software  and  during  the  transfer/test  portion, 
they  will  transfer  to  the  other  region.  The  basic  transfer  between  physical  contexts  will  remain 
the  same  as  described  previously  with  this  added  dimension.  We  started  negotiating  with 
Aptima  about  what  needs  to  be  changed  and  we  hope  that  they  will  be  able  to  deliver  the 
software  by  the  early  spring.  We  anticipate  being  done  running  teams  for  the  second  study  by 
the  end  of  summer  or  early  fall  and  completing  the  analysis  and  report  in  Fall  2005. 
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Exploring  the  Interaction  of  Implicit  and  Explicit  Processes  to 
Facilitate  Individual  Skill  Learning 

Contract#:  DASW01-00-K-0012  Contract  dates:  9/00-8/04 

Institution:  Rensselaer  Polytechnic  Institute  PI:  Ron  Sun 

Co-PI:  Robert  Mathews 

Research  Question(s)  -  How  can  learning  from  experience  and  learning  from  explicit  thinking 
(explicit  mental  models)  be  integrated  to  enhance  expertise? 

Technical  Barrier(s)  -  The  interaction  of  these  two  types  of  knowledge  is  not  yet  well 
understood,  more  experimental  data  are  needed,  and  complex  patterns  of  data  need  better 
explanation. 

Significance/Impact  for  Basic  Research  -  Understanding  how  implicitly  acquired  knowledge 
from  experience  interacts  with  explicitly  acquired  knowledge  (mental  models)  of  a  task  will  lead 
to  a  better  understanding  and  better  computational  theory/models  of  human  learning. 

Potential  transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Future  Force  Warrior  Training  program  (WP  215)  -  Training  programs  that  facilitate 
rapid  accurate  decision-making. 

•  Various  training  programs:  VICTOR:  Virtual  Individual  and  Collective  Training  for 
Future  Warriors  (WP  233);  SIMAIR:  Simulation-Focused  Collective  Aircrew  Training 
(WP  231),  and  Training  for  Interactive  Distributed  Environments  (WP  214)  -  How 
implicit  and  explicit  training  interact  and  affect  skill  learning. 

Overview 

Any  high  level  of  skill  depends  on  both  conceptual  and  subconceptual  (experiential) 
knowledge.  However,  experts  are  often  only  aware  of  their  explicit  conceptual  knowledge. 
Experientially  acquired  subconceptual  (implicit)  knowledge  is  more  akin  to  pattern  recognition. 
For  example,  when  you  recognize  a  friend’s  face,  you  instantly  know  who  the  person  is,  but  you 
may  not  be  aware  of  what  cues  or  features  are  being  used  to  recognize  him/her.  This  lack  of 
awareness  of  essential  implicit  experiential  knowledge  creates  serious  challenges  for  training  and 
learning  in  the  military  as  well  as  civilian  context.  The  overall  goal  is  to  better  understand  the 
interaction  of  the  two  types  of  knowledge.  The  specific  purposes  of  this  research  program  are  (1) 
to  experimentally  investigate  the  interaction  between  knowledge  gained  from  hands-on 
experiential  (implicit)  learning  and  explicitly  formulated  conceptual  knowledge  in  the 
development  of  skills  and  expertise,  (2)  to  clarify  how  conscious  reflection  and  resulting 
conceptual  knowledge  can  be  employed  to  enhance,  rather  than  interfere  with,  experiential 
learning,  (3)  to  discover  the  optimal  mix  of  experiential  and  conceptual  training  over  a  range  of 
skill  acquisition  from  early  novice  to  a  high  level  of  expertise,  and  (4)  to  develop  computational 
models  and  theories  of  human  skill  learning  that  emphasize  the  interaction  of  these  two  types  of 
learning,  which  may  be  of  use  in  structuring  and  optimizing  training,  on  the  basis  of  our 
experimental  and  simulation  results. 
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Research  Approach 

We  will  conduct  a  series  of  laboratory  experiments  in  two  different  task  domains: 
artificial  grammar  learning  and  process  control.  Both  tasks,  using  college  students  as 
participants,  involve  learning  a  system  that  operates  according  to  complex,  difficult-to-leam 
rules.  In  the  artificial  grammar  experiments,  participants  learn  to  spot  poison  can  labels  on  a 
computer  simulation  of  a  starship  that  has  been  invaded  by  enemy  agents.  Identification  of 
poison  food  labels  requires  learning  to  identify  sequences  of  letters  generated  by  a  finite  state 
grammar.  Training  for  this  task  might  include  memorizing  a  diagram  of  the  grammar  (explicit 
training),  memorizing  cases  (implicit  training),  or  an  integrated  training  that  involves  tracing 
cases  through  the  grammar  diagram. 

The  process  control  task  involves  learning  to  control  the  temperature  of  a  simulated 
nuclear  reactor  by  controlling  the  number  of  fuel  pellets.  The  appropriate  number  of  pellets  to 
use  depends  on  the  current  temperature  of  the  reactor  sometimes  creating  counterintuitive 
situations  where  increasing  the  number  of  fuel  pellets  decreases  temperature.  Also,  a  noise 
element  is  included  in  the  formula  making  the  results  somewhat  uncertain  over  trials.  This  task 
is  known  to  be  difficult  to  leam  and  difficult  to  explain  how  one  accomplishes  the  task  when  it  is 
learned. 

Accomplishments 

In  the  artificial  grammar  experiments,  we  found  slow  but  accurate  responding  among 
participants  when  explicitly  trained  and  fast  but  less  accurate  responding  when  implicitly  trained. 
Integrating  or  mixing  types  of  training  generally  produced  more  accurate  performance  than 
implicit  training  and  faster  performance  than  strict  explicit  training.  Simulations  using 
CLARION  have  confirmed  these  findings.  One  interesting  finding  was  that  when  exposed  to 
both  types  of  training,  participants  showed  a  tendency  to  prefer  using  the  implicit  mode.  In  the 
case  of  strict  explicit  followed  by  implicit  training  (a  pattern  that  is  common  in  many  training 
situations,  e.g.,  explicit  schooling  followed  by  field  training)  we  noticed  a  loss  of  accuracy  as 
learners  shifted  toward  the  implicit  mode.  We  think  training  can  be  accelerated  and  this  post¬ 
training  drop  eliminated  by  using  the  explicit  conceptual  knowledge  to  prime  rather  than 
compete  with  implicit  learning.  In  a  final  experiment  we  were  able  to  obtain  the  best  of  both 
worlds — fast  and  accurate  responding — by  incorporating  an  animated  version  of  a  diagram  of  the 
grammar  that  indicated  how  current  exemplars  fit  into  the  model  during  practice. 

The  process  control  results  show  that  detailed  reflection  during  task  performance 
interferes  with  learning.  Thus,  it  is  not  always  advantageous  to  combine  completely  explicit  and 
implicit  training.  However,  reflection  following  short  periods  of  practice  can  be  beneficial, 
especially  when  hints  are  provided  about  how  to  reflect.  Analyses  of  these  results  suggest  that 
reflecting  on  instances  where  one  responded  correctly  can  be  beneficial  to  learning.  However, 
more  general  reflecting  on  the  nature  of  the  task  or  general  rules  controlling  reactor  temperature 
do  not  seem  beneficial.  Simulations  using  CLARION  are  also  producing  interesting  hypotheses. 
Unlike  the  artificial  grammar  task,  there  are  large  individual  differences  in  the  ability  to  leam 
this  task.  Potentially,  we  can  facilitate  learning  in  these  “failing”  participants  by  instructing  them 
on  what  to  focus  on  when  they  reflect.  A  simple  hint  consisting  of  providing  three  correct 
responses  to  particular  task  situations  greatly  increased  learning. 
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In  the  process,  the  cognitive  architecture,  CLARION,  is  being  extended  and  refined.  It  is 
shaping  up  to  be  a  rather  comprehensive  framework  for  modeling  and  understanding  skill 
learning  in  a  variety  of  circumstances.  Cognitive  mechanisms,  such  as  working  memory, 
episodic  memory,  goal  structures,  semantic  memory,  implicit  associative  memory,  as  well  as 
motivational  and  meta-cognitive  constructs,  are  being  developed.  With  the  development  and 
specifications  of  these  mechanisms,  simulations  using  CLARION  have  provided  interesting 
interpretations  in  a  number  of  task  domains  and  more  simulations  will  be  under  way. 

Contributions  to  Basic  Research 

The  current  work  advances  basic  research  in  the  areas  of  learning  and  cognition.  One 
product  of  this  effort  is  a  conceptual  framework,  which  addresses  the  ways  these  two  types  of 
knowledge  interact  to  produce  expertise  (e.g.,  in  tasks  that  require  both  speed  and  accuracy), 
which  is  an  open,  but  important,  issue.  This  framework  (the  CLARION  model)  suggests  that 
performance  can  be  controlled  by  either  a  subconceptual  (connectionist)  knowledge  base 
(implicit  mode)  or  application  of  a  symbolic  conceptual  mental  model  (explicit  mode).  Implicit 
control  is  fast  but  prone  to  error,  particularly  in  early  levels  of  skill  acquisition.  Explicit  control 
is  more  accurate  but  slow  to  apply,  and  prone  to  loss  by  forgetting  over  a  retention  interval.  We 
have  found  that  reflection  about  how  one  is  performing  the  task  can  be  very  beneficial  following 
short  periods  of  practice.  However,  it  is  often  even  more  effective  when  learners  are  provided 
hints  that  direct  their  reflection  in  productive  directions.  These  are  important  findings  that 
advance  our  understanding  of  the  interaction  of  the  two  types  of  knowledge. 

A  computational  cognitive  architecture,  CLARION,  markedly  different  from  other 
existing  cognitive  architectures,  is  developed  in  this  work  to  simulate  and  capture  a  range  of 
quantitative  data  that  are  related  to  the  interaction,  based  on  the  above  ideas.  This  will  help  us  to 
explain  (eventually  to  predict)  training  and  learning  processes.  We  carried  out  simulation 
experiments  in  the  domains  of  process  control  tasks,  artificial  grammar  learning  tasks,  as  well  as 
some  other  tasks,  and  generated  new  insight  and  interpretations  that  can  further  explicate  the 
interaction  between  implicit  and  explicit  processes.  These  outcomes  (data,  models,  and  theories) 
provide  a  more  detailed,  clearer  and  more  comprehensive  perspective  on  skill  learning.  Our 
models  and  theories  will  be  useful  in  better  understanding  human  skill  learning,  as  well  as  in 
helping  to  improve  training  processes.  Our  models  and  theories  may  also  be  useful  in 
understanding  individual  differences  in  skill  learning  (based  on  the  implicit/explicit  interaction). 

Potential  Army/Military  Applications 

This  research  links  to  applied  programs  on  training  and  learning.  Specifically,  our  basic 
research  involves  understanding  learning  and  cognition  in  terms  of  the  interaction  between 
implicit  and  explicit  processes,  and  thus,  it  involves  testing  learning  and  training  principles  that 
may  be  used  to  enhance  human  learning.  The  goals  of  the  basic  research  align  with  those  of  the 
applied  programs.  Both  focus  on  training  effectiveness.  Hence,  work  from  our  basic  research 
could  inform  and  be  applicable  to  work  in  the  applied  programs.  For  example,  our  work  may 
offer  training  principles  that  enhance  training  effectiveness,  which  can  then  be  incorporated  into 
an  automated  tutoring  system.  There  are  of  course  many  other  ways  in  which  the  result  of  our 
basic  research  program  could  be  employed  in  the  applied  programs’  systems,  tools,  and  other 
products. 
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It  is  expected  that  applications  of  this  research  for  the  Army  will  be  in  the  area  of  training 
and  human  performance.  A  basic  problem  for  the  Army  is  how  to  ensure  that  novices  in  a 
Military  Occupational  Specialty  (MOS)  move  quickly  to  more  advanced  performance  (and 
perhaps  to  expertise)  as  a  result  of  their  training.  In  addition,  most  training  focuses  on  teaching 
conceptual  knowledge  rather  than  setting  up  the  opportunity  for  substantial  experiential 
(implicit)  knowledge.  While  this  may  be  appropriate  for  some  specialties,  some  other  specialties 
involve  working  with  complex  systems  that  are  better  learned  initially  through  extensive 
experience  than  with  lectures  or  textbook  lessons.  As  service  personnel  gain  in  expertise,  it  may 
often  become  appropriate  to  focus  more  on  conceptual  knowledge.  Our  research  will  lead  to 
some  initial  guidelines  about  1)  what  types  of  training  are  most  appropriate  for  learning  complex 
systems,  2)  what  types  of  training  are  most  appropriate  for  a  given  level  of  experience,  3)  how 
MOS  experts  might  best  communicate  their  knowledge  to  MOS  novices,  and  4)  individual 
differences  in  training  performance  and  how  these  can  be  explored  to  enhance  learning  (or  in 
other  words,  developing  individualized,  adaptive  training).  We  would  be  interested  in  talking 
with  our  liaison  officers  about  implementing  and  transitioning  these  ideas  in  the  future. 

Final  Summary 

The  results  of  our  experiments  support  our  theory/model  of  the  interactions  of  implicit 
and  explicit  learning  processes  during  skill  acquisition.  Strictly  implicit  training  is  effective  for 
fast  responding,  but  is  prone  to  error.  Strictly  explicit  training  results  in  slow,  but  accurate 
responding.  A  balance  of  both  worlds  (fast  and  accurate  responding)  can  be  obtained  by  using 
structural  models  in  training  that  emphasize  fast  but  accurate  responding.  Under  these  training 
conditions,  learners  acquire  the  ability  to  rely  on  implicit  knowledge  for  generating  an  initial 
sketch  of  a  solution  and  using  explicit  knowledge  to  fill  in  gaps  or  check  possible  errors. 

Our  research  also  demonstrates  that  implicitly  acquired  knowledge  can  be  much  more 
flexible  than  existing  research  suspected.  It  was  believed  that  implicitly  acquired  knowledge 
would  not  generalize  beyond  experienced  cases.  However,  we  found  that  people  could  acquire 
knowledge  from  artificial  grammar  cases  that  could  be  recombined  to  generate  a  wide  range  of 
valid  strings  not  yet  experienced.  This  form  of  learning  would  be  especially  valuable  if 
combined  with  external  help  that  could  correct  minor  errors,  such  as  the  computer  did  for  our 
participants  during  the  artificial  grammar  generation  task  (e.g.,  whenever  a  response  reached  an 
acceptable  level  it  was  corrected  by  the  computer). 

In  the  process  control  task,  learners  appear  to  acquire  correct  responses  more  from 
implicit  induction  rather  than  explicit  rule  generation.  In  fact,  our  college  level  participants  were 
particularly  bad  at  figuring  out  the  relatively  simple  equation  that  determined  reactor 
temperature,  even  when  they  were  given  hints  about  the  form  of  the  equation  and  a  calculator  to 
test  their  hypotheses.  Yet  a  simple  cue  that  includes  three  good  examples,  such  as,  “If  current 
temperature  is  10,000,  then  use  800  pellets”,  was  enormously  effective  in  enhancing  learning.  It 
appears  that  these  hints  showed  learners  how  to  look  at  the  task  in  terms  of  finding  good  cases. 
Reflective  thinking  in  between  practice  sessions  did  enhance  performance,  especially  when  the 
learner’s  reflection  was  focused  on  recalling  good  specific  cases  of  controlling  reactor 
temperature.  This  good  type  of  “recalling  instances”  reflection  could  be  induced  at  a  group  level 
as  well,  to  enhance  learning.  Therefore,  the  type  of  training  recommended  for  this  type  of 
complex  process  control  task  consists  of  short  periods  of  fast,  intense  practice  followed  by  short 


30 


intervals  of  reflection.  Also  providing  learners  of  a  few  examples  of  good  responses  to  specific 
situations  can  be  very  effective. 

These  results  should  be  further  developed,  because  they  may  have  significant 
implications  for  Army  training  and  for  other  applied  areas  of  Army.  The  knowledge  gained  from 
the  basic  research  would  apply  when  developing  training  programs,  with  a  better  understanding 
of  the  cognitive  processes  involved  in  skill  acquisition,  both  implicit  and  explicit,  and  when 
addressing  how  to  increase  training  effectiveness.  In  particular,  this  basic  research  program 
addresses  an  important  issue  when  developing  training  programs,  how  implicit  and  explicit 
processes  interact  and  impact  skill  learning  and  performance.  Much  more  work  is  needed  in  this 
area.  Similarly,  a  focus  of  research  in  decision  making  is  on  cognitive  skills  training  methods 
that  facilitate  rapid,  accurate  decision-making.  Although  this  is  a  different  focus,  the  current 
research  could  inform  the  applied  research  on  decision  making  of  an  important  consideration  in 
decision-making  —  implicit  and  explicit  cognitive  processes  and  their  interactions. 

Some  of  the  above  hypotheses  have  been  verified  through  computational  simulation 
using  CLARION.  In  particular,  the  CLARION  simulation  of  artificial  grammar  learning  has  led 
us  to  formulate  and  test  those  hypotheses  concerning  artificial  grammar  learning.  At  the  same 
time,  discrepancies  between  theoretical  models  and  experimental  data  have  led  to  new  designs  of 
further  human  experiments.  It  appears  that  CLARION  has  the  potential  to  be  a  comprehensive 
theory  of  a  range  of  psychological  tasks/domains.  Future  research  should  be  conducted  to 
further  develop  and  validate  this  approach. 
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Research  Question(s)  -  How  is  implicit  knowledge  acquired  from  experience,  integrated  with 
explicit  knowledge  from  analytical  thinking,  and  what  role  does  this  integration  play  in  skill 
learning  and  in  leading  to  better  skilled  performance? 

Technical  Barrier(s)  -  The  interaction  of  these  two  types  of  knowledge  is  still  poorly 
understood  and  more  important,  the  integration  of  these  two  types  of  knowledge  is  heretofore 
unexplored.  The  process  of  interaction  and  integration  appears  to  be  highly  complex  and  multi¬ 
faceted. 

Significance/Impact  for  Basic  Research  -  The  major  contribution  of  this  work  lies  in  coming 
up  with  a  significant  new  theory  that  explains  a  range  of  human  data  (learning  phenomena)  on 
the  basis  of  the  integration  of  implicit  and  explicit  knowledge. 

Potential  transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Future  Force  Warrior  Training  program  (WP  215)  -  Training  programs  that  facilitate 
rapid  accurate  decision-making. 

•  Various  training  programs:  VICTOR:  Virtual  Individual  and  Collective  Training  for 
Future  Warriors  (WP  233);  SIMAIR:  Simulation-Focused  Collective  Aircrew  Training 
(WP  231),  and  Training  for  Interactive  Distributed  Environments  (WP  214)  -  How 
implicit  and  explicit  training  interact  and  affect  skill  learning. 

Overview 

It  has  been  shown  that  high  level  of  skill  depends  on  both  conceptual  (explicit)  and 
subconceptual  (experiential,  implicit)  knowledge.  In  our  prior  work,  we  have  also  demonstrated 
that  there  is  often  a  lack  of  awareness  of  implicit  experiential  knowledge  and,  in  fact,  implicit 
knowledge  may  be  incompatible  with  explicit  knowledge.  These  two  problems  create  serious 
challenges  for  training  and  learning  in  the  military  as  well  as  civilian  context.  The  purpose  of 
this  research  program  is  to  better  understand  the  integration  of  these  two  types  of  knowledge  and 
how  their  integration  may  lead  to  better  skill  learning  and  performance.  Specifically,  we  will  (1) 
investigate  the  integration  of  implicit  knowledge  from  experiential  learning  and  explicitly 
formulated  conceptual  knowledge  in  the  development  of  skills  and  expertise,  (2)  explore  how 
conscious  reflection  and  resulting  conceptual  knowledge  may  be  integrated  with  implicit 
knowledge  to  enhance,  rather  than  interfere  with,  experiential  learning,  (3)  develop 
computational  models  and  theories  of  human  skill  learning  that  capture  the  fine-grained  details 
of  the  integration  of  these  two  types  of  learning,  which  may  be  of  use  in  improving  future 
training,  on  the  basis  of  our  experimental  results. 
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Research  Approach 

This  planned  project  investigates  the  integration  of  implicit  and  explicit  knowledge  in 
skill  learning.  In  part,  this  work  develops  further  a  cognitive  architecture  CLARION,  especially 
in  terms  of  capturing  of  process  details  of  bottom-up  learning  and  top-down  learning.  These 
processes  represent  hypothesized  ways  of  integrating  implicit  and  explicit  knowledge.  Equally 
important  are  a  set  of  human  experiments  to  explore  different  methods  of  facilitating  the 
integration  in  order  to  enhance  skill  acquisition. 

The  human  experiments  will  use  a  more  complex  process  control  task  than  those 
typically  used  in  the  implicit  learning  literature.  The  task  will  involve  several  interrelated  input 
and  output  variables.  The  added  complexity  should  require  learning  over  more  extended  periods 
of  time.  While  the  game  is  dynamic,  it  will  provide  feedback  by  discrete  time  steps.  That  is,  the 
participant  must  make  all  desired  input  variable  changes  before  entering  the  changes  (all  at  once) 
to  see  how  the  system  behaves.  This  division  of  performance  into  discrete  time  steps  will 
provide  the  opportunity  to  reveal  the  learners’  learning  strategies  and  view  changes  in  these 
learning  processes  over  time. 

In  addition  the  task  will  contain  a  special  model-building  screen  to  develop  and  reveal 
explicit  hypotheses  about  the  relationships  among  the  variables.  Participants  will  be  told  that 
their  goal  is  to  figure  out  the  pattern  of  relationships  among  input  and  output  variables  and 
correctly  map  these  relations  on  the  model-building  screen.  The  model-building  screen  will 
contain  icons  for  various  variables  at  both  current  and  next  time  steps.  Participants  can  connect 
icons  for  time-step  marked  variables  with  labeled  arrows  indicating  the  strength  (1-10)  and 
direction  (+  or  -)  of  their  relations.  The  screen  will  also  allow  use  of  two  different  colors  on  the 
arrows  connecting  variables,  black  for  relations  considered  definite  and  red  for  hypothetical,  but 
as  yet  unproven,  relations.  The  task  is  finished  when  the  participant  builds  a  model  of  the  task 
sufficiently  complete  to  control  performance  of  the  system  within  a  criterion  range  (e.g.,  90% 
correct). 

In  various  experiments,  opportunities  to  switch  from  the  play  screen  to  the  model 
building  screen  will  be  controlled.  In  some  experiments,  participants  will  play  the  game  at  a  fast 
pace  (must  respond  every  six  seconds)  to  build  implicit  (without  explicit)  knowledge  of  the 
game.  Then,  in  a  second  phase  of  the  experiment,  they  will  be  allowed  access  to  both  screens  to 
study  how  they  translate  their  implicit  knowledge  of  the  game  into  an  explicit  model  of  the  task. 
In  other  experiments,  people  will  be  encouraged  to  engage  explicit  learning  processes  to  learn 
how  to  play  the  game.  They  will  be  required  to  formulate  and  test  their  hypotheses  by  switching 
back  and  forth  from  the  model  screen  to  the  game  screen.  These  experiments  will  provide  us 
with  snapshots  of  how  people  attempt  to  implicitly  and  explicitly  learn  the  task.  Other 
experiments  will  test  various  ways  of  integrated  training  such  as  having  an  animated  version  of 
the  model  building  screen  on  the  game  playing  screen.  The  animated  model  will  highlight 
correct  (anticipated)  and  incorrect  (unanticipated)  changes  in  the  system.  In  simpler  domains  we 
have  found  that  such  animated  diagrams  can  enhance  both  speed  and  accuracy  of  performance. 
Other  variables  studied  will  include  various  hints  to  help  build  accurate  models  and  whether  task 
feedback  is  performance  (error  detection)  focused  versus  discovery  (creativity)  focused.  The 
later  variable  is  expected  to  interact  with  individual  differences  (see  Kluger  and  Van-Dijk,  and 
Erez;  ARI  Contract  #  DASW01-02-K-705 12). 
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Accomplishments  -  N/A  -  New  Start. 

Contributions  to  Basic  Research 

The  major  contribution  of  this  work  lies  in  coming  up  with  a  significant  new  theory  that 
explains  a  range  of  human  data  (learning  phenomena)  on  the  basis  of  the  integration  of  implicit 
and  explicit  knowledge.  One  product  of  this  effort  is  a  conceptual  framework,  which  addresses 
the  ways  these  two  types  of  knowledge  may  be  integrated  to  produce  expertise.  This  framework 
(the  CLARION  model)  suggests  that  performance  can  be  controlled  by  either  a  subconceptual 
(connectionist)  knowledge  base  (implicit  mode)  or  application  of  a  symbolic  conceptual  mental 
model  (explicit  mode).  Implicit  control  is  fast  but  prone  to  error,  particularly  in  early  levels  of 
skill  acquisition.  Explicit  control  is  more  accurate  but  slow  to  apply,  and  prone  to  loss  by 
forgetting  over  a  retention  interval.  Their  integration  may  lead  to  better  performance. 

A  computational  cognitive  architecture,  CLARION,  which  is  already  different  from  other 
existing  cognitive  architectures,  will  be  further  developed  in  this  work  to  simulate  and  capture  a 
range  of  quantitative  data  that  are  related  to  the  integration  of  the  two  types  of  knowledge,  based 
on  the  above  ideas.  This  will  help  us  to  explain  (eventually  to  predict)  training  and  learning 
processes.  We  will  carry  out  simulations  to  generate  new  insight  and  interpretations  that  can 
further  explicate  the  integration  of  implicit  and  explicit  processes.  These  outcomes  (data, 
models,  and  theories)  provide  a  new  perspective  on  skill  learning.  Our  models  and  theories  will 
be  useful  in  better  understanding  human  skill  learning  as  well  as  in  helping  to  improve  training 
processes. 

Potential  Army/Military  Applications 

This  research  links  to  applied  programs  on  training  and  learning.  Specifically,  our  basic 
research  involves  understanding  learning  and  cognition  in  terms  of  the  integration  of  implicit  and 
explicit  processes,  and  thus  it  involves  testing  learning  and  training  principles  that  may  be  used 
to  enhance  human  learning.  The  goals  of  the  basic  research  align  with  those  of  the  applied 
programs.  Both  focus  on  training  effectiveness.  Hence,  work  from  our  basic  research  could 
inform  and  be  applicable  to  work  in  the  applied  programs.  For  example,  our  work  may  offer 
training  principles  that  enhance  training  effectiveness,  which  can  then  be  incorporated  into  an 
automated  tutoring  system.  There  are  of  course  many  other  ways  in  which  the  results  of  our  basic 
research  program  could  be  employed  in  the  applied  programs’  systems,  tools,  and  other  products. 

We  expect  that  Army  applications  of  this  research  will  be  in  the  area  of  training  and 
human  performance.  A  basic  problem  for  the  Army  is  how  to  ensure  that  novices  in  a  Military 
Occupational  Specialty  (MOS)  move  quickly  to  more  advanced  performance  (and  perhaps  to 
expertise)  as  a  result  of  their  training.  In  addition,  most  training  focuses  on  teaching  conceptual 
knowledge  rather  than  setting  up  the  opportunity  for  substantial  experiential  (implicit) 
knowledge.  While  this  may  be  appropriate  for  some  specialties,  some  other  specialties  involve 
working  with  complex  systems  that  are  better  learned  initially  through  extensive  experience  than 
with  lectures  or  textbook  lessons.  As  service  personnel  gain  in  expertise,  it  may  often  become 
appropriate  to  focus  more  on  conceptual  knowledge.  In  this  process,  the  integration  of  the  two 
kinds  of  knowledge  is  expected  to  be  highly  important.  Our  research  will  lead  to  some  initial 
guidelines  about  how  to  best  facilitate  knowledge  integration  and  thus  learning  and  performance 
improvements. 
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Future  Plans 

Our  contract  will  begin  towards  the  end  of  FiscalYear  2005.  We  will  first  acquire  new 
equipments  and  develop  software  for  conducting  human  experiments.  Then  on  that  basis,  we 
will  begin  to  conduct  the  first  set  of  experiments  as  specified  in  our  proposal. 

Simultaneously,  we  will  develop  the  basic  outline  of  a  computational  model  for  capturing 
various  methods  for  integrating  implicit  and  explicit  knowledge  in  skill  learning,  namely  various 
bottom-up  and  top-down  learning  algorithms.  We  will  then  test  them  against  human  data  to  be 
generated  from  human  experiments.  We  will  also  work  on  further  cognitive  architecture 
developments. 

We  will  start  to  prepare  manuscripts  of  theoretical  and/or  experimental  papers,  arguing 
and  developing  our  unique  approach. 
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RACO  RESEARCH  OBJECTIVE  #2: 


Provide  fundamental  knowledge  to  improve  leader  and  team 
performance. 


Research  that  falls  under  this  heading  provides  concepts  and  methods  for 
accelerating  leader  development,  understanding  and  developing  leader  adaptability 
and  flexibility,  and  discovering  and  testing  the  basic  cognitive  principles  that 
underlie  effective  leader-team  performance.  Understanding  the  dynamics  of  small 
group  leadership  in  face-to-face  and  distributed  team  environments  is  critical  to 
this  research  objective  as  well. 
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The  Leadership  Formula:  P  x  M  x  D 


Contract  #:  DASW01-01-K-0004  Contract  Dates:  09/01/01-03/01/2004 

Institution:  The  Center  for  Outstanding  PI:  Dr.  Reuven  Gal 

Leadership  Co-PI:  Dr.  Micha  Popper 

Problem(s)/Research  Question(s)  -  1)  How  leaders  differ  from  non-leaders  in  terms  of 
potential  (P  -  psychological  capacities),  motivation  (M)  and  developmental  experiences  (D)  from 
early  childhood;  2)  How  the  variables  in  the  P.M.D  formula  affect  the  emergence  of  leadership. 

Technical  Barrier(s)  -  Not  much  is  known  about  how  leaders  develop.  The  literature-includes 
mostly  biographies  and  psycho-biographies,  and  some  psychological  studies  based  on 
retrospective  recollections  of  leaders.  The  lack  of  psychological  studies  on  leadership  development 
stems  from  absence  of  models  and  empirically  testable  conceptualizations. 

Significance/Impact  for  Basic  Research  -  The  research  addresses  three  leadership  concepts 
theoretically  and  empirically.  These  concepts  are  leadership  potential,  motivation  to  lead,  and 
leadership  development. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force  Soldier  (ATO 
IV.HS.2002.01) 

•  Accelerating  Leader  Development  (WP  269, 295,  and  ATO:  III.HS.2004.01) 

•  Assessing  and  Developing  Leaders  for  Special  Operations  Units  (WP  218). 

Overview 

The  conceptual  framework  underlying  the  study  suggests  that  leadership  is  a  function  of  a 
given  potential  (P),  relevant  motivation  (M),  and  certain  developmental  processes  (D).  These 
three  components  comprise  the  “Leadership  formula.”  This  formula  provides  a  theoretical 
framework  within  which  internal  aspects,  motivational  needs  and  developmental  processes  are 
studied  as  antecedents  of  leadership.  Our  argument  is  that  those  who  possess  certain 
psychological  capacities  (P)  (e.g.,  high  self-efficacy,  or  a  low  level  of  anxiety),  have  a  strong 
motivation  to  lead,  have  undergone  relevant  developmental  processes  and  are  “equipped”  with 
certain  developmental  dispositions  can  become  leaders. 

Our  research  hypotheses  are: 

1 .  “Leaders”  will  have  a  higher  potential  (certain  psychological  capacities)  than  “non-leaders.” 

2.  “Leaders”  will  have  a  higher  motivation  to  lead  than  “non-leaders.” 

2a.  “Leaders”  will  have  a  different  motivational  profile  from  “non-leaders.” 

3a.  “Leaders”  will  report  more  “leadership-shaping  experiences”  (LSE)  than 
“non-leaders.” 

3b.  “Leaders”  will  experience  different  “leadership  -shaping  experiences”  than 
“non-leaders.” 

3c.  “Leaders”  will  be  characterized  by  higher  levels  of  “openness”  to  new 
experiences  than  “non-leaders.” 
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Research  Approach 

This  research  is  based  on  a  comparative  examination  of  several  samples  of  IDF  soldiers 
(from  Infantry  and  Armored  units)  undergoing  their  basic-training.  By  distinguishing  “leaders” 
from  “non-leaders”  and  subsequently  assessing  various  P,  M  and  D  measurements,  this  field 
study  enabled  us  to  examine  our  three  categories  of  hypotheses  relating  to  the  leadership 
formula:  Potential,  Motivation  and  Development.  The  questionnaires  were  administered  to  the 
soldiers  in  two  phases:  at  the  beginning  of  their  basic  training  and  after  a  year  of  service.  In  the 
second  phase,  we  returned  to  our  original  subjects  and  administered  the  three  questionnaires  that 
measure  those  attributes  that  are  conceptually  dynamic  and  may  change  over  time  (General  Self- 
efficacy,  Motivation  to  Lead,  and  Openness  to  Experience). 

Accomplishments 

We  obtained  questionnaires  from  402  IDF  soldiers  and  conducted  60  LSE  interviews.  We 
have  finished  analyzing  the  questionnaires  and  the  results  so  far  indicate  that: 

•  Potential  to  lead  (P):  All  the  potential  variables  differentiate  significantly  between  “leaders” 
and  “non-leaders.”  An  article  in  Military  Psychology  based  on  these  results  is  currently  in  press 
(Popper,  M.,  Amit,  K.,  Gal,  R.,  Sinai,  M.  &  Lissak,  A.,  2004.  The  capacity  to  lead:  Major 
psychological  differences  between  “leaders”  and  “non-leaders.”  Military  Psychology,  16,  4,  245- 
263). 

•  Motivation  to  lead  (Ml:  “Leaders”  have  higher  motivation  to  lead  than  “non-leaders.”  Four 
motivation  to  lead  components  significantly  differentiate  between  “leaders”  and  “non-leaders.” 
An  academic  journal  paper,  based  on  these  results,  is  under  review  in  Military  Psychology. 

•  Leadership  Development:  “Leaders”  were  found  to  be  characterized  by  higher  levels  of 
“openness”  (the  reason  for  this  was  also  studied)  and  report  more  “leadership-shaping 
experiences”  (LSE)  than  “non-leaders.”  The  results  were  analyzed  quantitatively  and 
qualitatively  to  study  the  types  of  experiences  that  affect  leader  development.  An  academic 
article  based  on  the  quantitative  results  will  be  submitted  soon  to  Leadership  Quarterly. 

•  We  are  now  analyzing  the  qualitative  data  concerning  the  development  issues. 

Contributions  to  Basic  Science 

This  investigation  will  contribute  to  basic  research  as  it  provides  a  coherent  and 
integrative  framework  for  dealing  with  the  following  questions:  What  kind  of  individuals 
become  leaders?  What  is  their  psychological  uniqueness  and  how  is  it  formed  from  early 
childhood?  The  developmental  aspects  in  particular  are  known  as  an  uncharted  area  waiting  to 
be  studied.  Specifically,  our  research  focused  on  three  components:  Leadership  potential  (P), 
motivation  to  lead  (M),  and  leadership  development  (D).  We  examined  these  in  our  research, 
using  existing  psychological  instruments  to  test  the  P  component,  and  constructing  new 
instruments  to  test  the  M  and  D  components.  We  expanded  the  concept  of  “motivation-to-lead,” 
a  concept  scarcely  examined  in  the  literature,  both  substantially  and  technically.  We  constructed 
three  developmental-leadership  assessment  tools:  Openness  to  Experience  questionnaire,  LSE 
interview  and  LSE  questionnaire.  Finally,  we  constructed  a  questionnaire  identifying  “leaders” 
and  “non-leaders.” 
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Potential  Army/Military  Applications 

This  research  was  conducted  in  the  Israeli  Defense  Force  (IDF).  Unlike  civilian 
organizations  where  leadership  is  diffused  in  the  manager’s  functions  and  formal  authority,  in  the 
military,  pure  leadership  is  much  more  prevalent  and,  in  fact,  is  an  essential  driving  force  that 
generates  the  soldiers’  readiness  to  risk  their  lives. 

Furthermore,  within  the  military  the  distinction  between  “leaders”  and  “non-leaders”  is  much 
more  evident  than  in  most  other  institutions.  This  distinction  is  somewhat  contaminated, 
however,  in  those  military  organizations  (such  as  the  U.S  Army)  where  the  distinction  between 
officers  and  non-officers  is  made  “institutionally”  (through  military  academies),  rather  than 
“naturally”  (through  a  gradual  selection  process).  A  salient  example  of  the  latter  type  is  the  IDF. 
Starting  on  an  equal  footing,  all  Israeli  conscripts  undergo  an  ongoing  selection  process  during 
their  initial  training  period,  and  out  of  this  prolonged  process  emerge  those  who  eventually 
become  officers  or  NCO’s.  This  unique  situation  makes  the  IDF  an  extraordinary  natural 
laboratory  for  examining  leadership  development.  Moreover,  this  field  study  was  conducted  on 
two  different  army  corps,  Infantry  and  Armored  Corps,  in  order  to  examine  whether  they  reflect 
different  leadership-forming  processes. 

Overall,  our  research  may  provide  a  unique  opportunity  to  understand  how  “leaders” 
differ  from  “non-leaders”  and  what  makes  pure  leadership.  Furthermore,  our  research  may 
unravel  some  of  the  mysteries  connected  with  the  nature-nurture  origins  of  leadership,  as  well  as 
suggesting  some  important  and  practical  implications  related  to  leader  selection  and  development 
(i.e.  training)  in  any  modem  military. 

Final  Summary 

In  all  three  components  (P.M.D)  of  the  leadership  formula,  differences  were  found 
between  leaders  and  non-leaders.  Leaders  were  found  to  have  a  lower  level  of  trait  anxiety, 
higher  levels  of  openness,  self-efficacy  and  optimism,  and  to  be  characterized  more  than  non¬ 
leaders  by  an  internal  locus  of  control  and  a  secure  attachment  style.  In  distinguishing  between 
“fundamental  components”  (e.g.,  openness,  and  optimism,  locus  of  control  -  aspects  that  are 
close  to  the  notion  of  “disposition”)  and  “learned  components”  (e.g.,  self-efficacy),  we  found 
that  fundamental  components  affect  the  mode  and  intensity  of  a  leader’s  development.  The  types 
of  relationship  between  these  components  (interactive  or  additive)  are  explored  and  discussed. 

The  hypotheses  were  derived  from  a  conceptual  framework  that  was  formulated  on  the 
basis  of  various  leading  psychological  theories  in  the  domains  of  personality  (e.g.,  anxiety, 
openness),  developmental  psychology  (e.g.,  attachment  theory),  and  theories  of  motivation  (e.g., 
expectancy  theory).  The  findings  indicate  that  the  “integrative  package”  that  was  constructed  has 
a  significant  theoretical  contribution  as  it  combines  different  psychological  views  and  serves  as  a 
framework  for  creating  testable  concepts  in  a  field  known  as  a  void.  Besides  its  theoretical 
contribution,  this  research  has  direct  applications  in  the  domain  of  selecting,  assessing  and 
developing  leaders,  particularly  at  early  stages  of  their  lives. 
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Leadership:  Enhancing  Team  Adaptability  in  Dynamic  Settings 

Contract  Dates:  01/01/03-09/30/06 
PI:  Dr.  Katherine  J.  Klein 
Co-PIs:  Dr.  Steve  W.  J.  Kozlowski 
Dr.  YanXiao 

Liaisons:  Stan  Halpin,  LDRU  & 

Scott  Graham,  IFRU 

Problem(s)/Research  Question(s)  -  What  predicts  individuals’  adaptive  performance  in 
dynamic  settings?  What  predicts  teams '  adaptive  performance  in  dynamic  settings?  What  can 
team  leaders  do  to  enhance  adaptive  individual  and  team  performance? 

Technical  Barrier(s)  -  While  research  on  leadership  is  abundant,  as  is  research  on  teams, 
surprisingly  few  researchers  have  focused  specifically  on  team  leadership.  Further,  it  is 
challenging  to  identify,  gain  access  to,  study,  and/or  stimulate  truly  dynamic  work  settings. 

Significance/Impact  for  Basic  Research  -  Our  findings  will  help  to  clarify  the  nature  artd 
antecedents  of  adaptive  individual  and  team  performance  in  dynamic  settings,  expanding 
research  on  (1)  leadership;  (2)  team  effectiveness;  and  (3)  dynamic  work  contexts. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  maybe  usefully  employed  in: 

•  LEADERSHIP:  Developing  Leaders  in  a  Changing  Army  (WP  103) 

•  Future  Force  W ardor  Training  (WP  215) 

•  Assessing  and  Developing  Leaders  Within  Special  Operations  Units  (WP  2 1 8) 

•  Accelerating  Leader  Development  (WP  269);  Accelerating  Leader  Development  (WP 
285) 

•  SELECT21:  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force 
Soldier  (WP  257)  and  ATO:  IV.HS.2002.01 . 

•  FUTURE  TRAIN:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  Combat 
Team  Commanders  and  Staffs  (WP  211) 

Overview 

Our  research  is  designed  to  illuminate  the  influence  of  individual  differences,  team 
characteristics,  and  leadership  on  individual  and  team  performance  in  dynamic  work 
environments.  Dynamic  settings  are  work  environments  in  which: 

1 .  Diverse  specialists  work  in  teams  to  perform  highly  interdependent  tasks; 

2.  Tasks  are  high  in  novelty,  uncertainty,  and  urgency; 

3.  The  pace  of  work  is  varied  and  unpredictable  as  emergency  events  occur 
frequently  but  at  unpredictable  times; 

4.  Team  composition  (i.e.,  team  membership)  changes  frequently,  such  that  team 
members  may  not  know  many  of  the  individuals  on  their  team;  and 

5.  Effective  task  performance  is  dependent  on  team  member  learning,  development, 
and  adaptation. 
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Theory  and  research  regarding  the  factors  that  allow  individuals  and  teams  to  thrive  in 
such  settings  are  extremely  limited.  Our  research  is  designed  to  begin  to  fill  this  gap  in  the 
literature.  Ultimately,  our  work  will  inform  training  specifications  for  leader  development  so  that 
Army  leaders  are  equipped  with  the  skills  needed  to  shape  and  influence  team  learning, 
development,  and  adaptability. 

Research  Approach 

Our  research  approach  is  to  use  diverse,  complementary  research  strategies  to  gain  new 
understanding  of  the  factors  that  influence  individual  and  team  adaptive  performance  in  dynamic 
settings.  We  have  begun  to  conduct  new  qualitative  (primarily  interview)  data  in  a  real-life, 
dynamic  work  setting:  The  Maryland  Shock  Trauma  Center  (STC).  STC  is  the  hub  of 
Maryland’s  system  of  emergency  trauma  care,  treating  over  6,000  victims  of  traumatic  injury 
each  year.  We  are  using  our  qualitative  findings  to  refine  our  theoretical  model  and  to  develop 
survey  measures  of  the  antecedents  of  individual  and  team  adaptive  performance. 

We  complement  the  real-world  insights  that  we  gain  in  the  STC  with  rigorous, 
experimental  research  in  the  MSU  ADAPT  Lab  (Accelerated  Development  and  Adaptive 
Performance  Training).  The  experimental  work  is  based  on  the  theoretical  foundation  of  self¬ 
regulation  as  a  means  to  examine  the  psychological  mechanisms  underlying  learning,  motivation, 
and  performance.  The  primary  focus  of  the  lab  research  is  to  identify  and  develop  techniques  that 
leaders  can  use  to  enhance  individual  learning,  team  development,  and  adaptability. 

Accomplishments:  Research  in  the  Shock  Trauma  Center 

We  wrote  a  qualitative  case  study  and  analysis  of  the  team  leadership  system  that  allows 
teams  of  attending  physicians,  residents,  nurses  and  technicians  to  provide  reliable  and  effective 
care  to  Shock  Trauma  patients  (specifically  patients  in  the  Trauma  Resuscitation  Unit,  or  TRU). 
This  paper  received  a  revise-and-resubmit  from  Administrative  Science  Quarterly  and  we  are 
working  on  the  revision.  In  our  newest  data  collection  effort,  we  have  conducted  semi-structured 
interviews,  focusing  specifically  on  the  experiences  of  surgical  and  emergency  medicine 
residents  and  fellows.  Our  interviews  were  designed  to  provide  preliminary  answers  to  the 
following  questions 

What  constitutes  adaptive  individual  performance  in  the  STC?  Interviewees 
emphasized  that  two  criteria  are  most  indicative  of  adaptive  individual  performance:  learning 
and  team  citizenship  behaviors.  Individuals  who  acquire  new  skills  in  performing  medical  and 
surgical  procedures  and  who  provide  assistance  to  other  team  members’  facilitate  their  team’s 
adaptation  to  unpredictable  work  demands. 

What  individual  differences  predict  adaptive  individual  performance  in  the  STC? 

Interviewees’  comments  suggest  that  residents  and  fellows  are  most  likely  to  gain  new  skills 
during  their  rotations  within  the  STC  if  they  have  a  strong  learning  orientation  (i.e.,  they  are 
eager  to  acquire  new  skills  and  willing  and  able  to  learn  from  failure  and  from  both  positive  and 
negative  feedback).  Team  member  goals  are  also  likely  to  be  influential.  Team  members  who 
expect  to  work  in  emergency  medicine  or  trauma  care  -  and  thus  regard  their  STC  rotations  as 
instrumental  of  their  long-term  career  effectiveness  -  are  most  likely  to  learn  a  lot  during  their 
rotations.  Finally,  team  member  confidence,  or  self-efficacy,  is  critical  as  they  face  novel 
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challenges  in  treating  patients’  suffering  injuries  that  the  residents  and  fellows  have  not  treated 
before. 


Interviewees’  comments  suggest  that  personality  and  values  may  predict  individuals’ 
team  citizenship  behaviors  best.  Team  members  who  are  conscientious,  agreeable,  extraverted, 
and  emotionally  stable  seem  most  likely  to  be  attentive  to  opportunities  to  assist  other  team 
members’  in  carrying  out  their  work,  facilitating  the  work  of  the  entire  team.  Further,  team 
members  who  have  a  strong  work  ethic,  or  preference  for  activity,  may  also  be  particularly  high 
in  team  citizenship  behaviors. 

In  their  comments,  interviewees  highlighted  two  different  strategies  for  managing  the 
transition  to  a  novel  work  setting:  Observation  and  proactive  feedback  seeking.  That  is,  some 
interviewees  emphasized  that  they  learned  appropriate  behaviors  in  the  new  setting  by  hanging 
back  and  observing  others.  In  contrast,  others  emphasized  that  they  actively  sought  out  others  of 
all  ranks  and  asked  them  for  information  and  guidance. 

These  qualitative  findings  underscore  and  mirror  several  of  the  key  findings  emerging 
from  Kozlowski’s  research  in  the  MSU  ADAPT  Lab. 

What  constitutes  adaptive  team  performance  in  the  STC?  Interviewees’  answers  to 
questions  addressing  this  topic  reinforced  the  importance  of  learning  and  of  team  citizenship 
behavior.  Interviewees  emphasized  that  effective,  adaptive  teams  are  ones  in  which  more 
experienced  team  members  teach  less  experienced  team  members  to  perform  critical  emergency 
medical  and  surgical  procedures.  Further,  interviewees  also  emphasized  that  effective,  adaptive 
teams  are  ones  in  which  team  members  come  to  each  other’s  assistance,  helping  one  another  to 
perform  both  the  most  and  the  least  glamorous  of  team  tasks.  Interviewees’  comments  suggest 
that  adaptive,  high  performing  teams  are  distinguished  by  their  dense  social  networks  in  which 
team  members  are  linked  by  learning,  teaching,  and  mutual  assistance  ties. 

What  can  team  leaders  do  to  enhance  individual  and  team  adaptive  performance  in 
the  STC?  Interviewees  identified  a  number  of  functions  that  leaders  may  perform  to  enhance 
individual  and  team  performance.  These  include  communication  and  providing  initiating 
structure  to  clarify  procedures,  plans,  and  norms  and  to  ensure  that  team  members  who  vary  in 
experience  and  expertise  are  “all  on  the  same  page.”  Interviewees  also  value  leaders  who  coach 
and  teach  team  members,  enhancing  team  members’  skills  and  reinforcing  their  learning 
orientation.  Further,  leaders  who  are  calm,  well-connected  to  sources  of  information  and 
resources  external  to  the  team,  and  who  facilitate  shared  learning  and  socialization  are 
particularly  effective,  interviewees  reported. 

Accomplishments:  Research  in  the  ADAPT  Lab 

We  developed  a  conceptual  framework  that  specifies  leader  functions  which  leverage 
self-  and  team-regulation.  Further,  we  conducted  new  experiments  to  develop  and  test  our  ideas 
regarding  the  antecedents  of  individual  self-regulation  and  learning. 

What  can  team  leaders  do  to  enhance  individual  and  team  adaptive  performance? 

Our  conceptual  framework  integrates  the  functional  team  leadership  literature  with  work  that 
adopts  a  dynamic  perspective  on  team  processes,  performance,  and  development.  It  is  formulated 


43 


around  two  dynamic  processes — (a)  environmentally-driven  team  task  cycles  and  (b)  the 
evolution  of  team  development — that  create  contingencies  for  aligning  leader  behavior.  The 
framework  specifies  how  environmental  inputs  create  variations  in  task  complexity,  and  how 
episodic  team  task  cycles  can  be  meshed  with  leadership  functions  of  preparation,  action, 
monitor/intervention,  and  reflection.  By  harnessing  naturally  occurring  task  cycles  and  applying 
the  functions,  leaders  can  leverage  self-regulatory  processes  that  underlie  learning  to  build 
targeted  skill  sets.  As  team  member  skills  improve  and  crystallize,  it  enables  the  leader  to 
transition  the  team  to  more  complex  skills  and  capabilities.  The  result  of  this  developmental 
process  is  a  self-regulating,  continuously  improving,  and  adaptive  team.  The  framework  provides 
a  basis  for  specifying  team  leader  actions  to  fit  shifting,  episodic,  and  evolving  situational 
demands,  and  has  implications  for  research  on  leadership,  leader  training,  and  team  development. 

How  can  leaders  frame  the  situation  to  enhance  self-regulation?  Ongoing  research  in 
the  ADAPT  Lab  (supported  in  part  by  this  project)  demonstrates  that  an  individual’s  orientation 
toward  complex  skill  acquisition  (learning  or  mastery  vs.  performance)  has  important  effects  on 
learning,  performance,  and  performance  adaptability;  a  learning  or  mastery  orientation  is 
superior  for  complex  skill  acquisition.  A  learning  orientation  can  be  induced  by  manipulating 
cues  and  instructions  that  emphasize  skill  mastery,  learning  from  errors,  and  exploration, 
whereas  a  performance  orientation  can  be  induced  by  an  emphasis  on  accomplishing 
performance  goals,  minimizing  mistakes,  and  demonstrating  proficiency.  These  inductions  can 
be  delivered  by  leader  directives.  This  is  important  because  such  inductions  are  easily  created 
and  can  (inadvertently)  contradict  training  goals.  For  example,  training  is  typically  predicated  on 
explicit  goals  to  leam  specific  skills.  If  leaders  also  emphasize  high  errorless  performance  during 
training  (performance  frame),  they  induce  a  performance  orientation  that  degrades  self¬ 
regulation  and  learning.  This  work  has  implications  for  specifying  leadership  functions  and 
training  design  characteristics  so  as  enhance  self-regulation,  learning,  and  adaptability. 

Contributions  to  Basic  Science 

Our  research  will  illuminate  the  psychological  mechanisms  underlying  individual  and 
team  self-regulation  in  dynamic  work  settings.  Further,  our  research  will  identify  leader 
behaviors  and  functions  (e.g.,  goal  clarification,  feedback,  coaching)  that  enhance  individual  and 
team  self-regulation  and  thereby  enhance  individual  and  team  learning  and  development  in 
dynamic  settings.  Current  research  on  leadership  and  on  individual  and  team  performance  is 
dominated  by  relatively  static,  structural  approaches.  Our  research  highlights  the  dynamic  nature 
of  individual,  team,  and  leader  performance  required  in  dynamic  work  settings. 

Potential  Army  /  Military  Applications 

Our  research  findings  have  implications  for  military  leadership,  training,  selection,  and 
team  (e.g.,  platoon)  design.  The  knowledge  we  are  generating  can  be  applied  to  improve 
leadership  (in  action),  leader  development  (training  specifications  for  leader  capabilities),  and 
training  design  and  delivery  (what  we  develop  can  be  delivered  by  leaders  in  action  or  by 
technology-based  training  systems).  Our  findings  can  also  inform  the  selection  of  soldiers  for 
work  in  dynamic  team  settings,  shedding  light  on  individual  differences  that  predict  effective 
learning  and  performance  in  dynamic  settings.  Finally,  analyses  of  the  effects  of  team  social 
networks  in  the  Shock  Trauma  Center  may  inform  team  design,  clarifying  the  influence  of  the 
nature  and  timing  of  changes  in  team  membership  on  team  norms  and  subsequent  performance. 
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Future  Plans 


During  the  next  year,  we  will  continue  research  in  the  Shock  Trauma  Center  and  the 
ADAPT  Lab  in  an  effort  to  continue  to  identify  individual,  team,  and  leader  characteristics  that 
enhance  individual  and  team  adaptive  performance  in  dynamic  settings.  Specific  goals  of  our 
research  including: 

•  Drawing  on  the  findings  from  our  qualitative  field-research  in  the  Shock  Trauma  Center 
to  inform  and  influence  research  in  the  MSU  ADAPT  Lab  and  vice-versa. 

•  Conducting  survey  research  in  the  Shock  Trauma  Center  to  test  hypotheses  regarding  (a) 
individual-level  antecedents  of  performance  in  the  STC  (e.g.,  learning  versus 
performance  orientation);  (b)  the  influence  of  leader  behaviors  on  team  member  learning 
and  performance;  and  (c)  the  influence  of  team  norms  and  social  network  structures  on 
individual  and  team  learning  and  performance. 

•  Conducting  experimental  research  in  the  MSU  ADAPT  Lab  to  test  hypotheses  regarding 
(a)  the  effects  of  leader  behaviors  (mastery  vs.  performance  inductions)  on  kinds  and 
amount  of  feedback  sought  by  team  members  in  a  simulated  dynamic  work  environment; 
and  (b)  the  effects  of  feedback  on  team  members’  goal  orientation  states.  (In  brief, 
mastery  inductions  are  expected  to  yield  more  feedback  seeking  overall  and  more 
feedback  seeking  focused  on  self-improvement,  process  orientated,  and  deeper  task 
strategies.  In  contrast,  performance  inductions  are  expected  to  yield  less  feedback 
seeking,  with  feedback  choices  related  to  normative  comparisons,  outcome  orientated, 
and  surface  task  information.) 
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Self-Initiated  Development  of  Leadership  Capabilities: 

Toward  Establishing  the  Validity  of  Key  Motivational  Constructs 

and  Assessment  Tools 

Contract  #:  W74V8H-05-K-0001  Contract  Dates:  1 1/1/04-10/31/07 

Institution:  Georgia  State  University  P.I.:  Todd  J.  Maurer 

Liaison:  Larry  Laffitte,  LDRU 

Problem(s)/Research  Question(s)  -  The  problem  addressed  in  this  research  is  to  identify  and 
validate  new  individual  difference  characteristics  that  are  relevant  to  motivating  leadership 
development,  as  well  as  ways  of  measuring  those  differences  in  people. 

Technical  Barrier(s)  -  Little  is  known  about  predicting  leadership  development  behavior 
because  most  research  has  been  directed  at  predicting  the  performance  of  leaders. 

Significance/Impact  for  Basic  Research  -  New  behavioral  constructs  are  needed  that  will  be 
useful  in  understanding,  predicting  and  ultimately  enhancing  leadership  development.  It  is 
expected  that  hypothesized  relationships  of  these  leadership  development  constructs/measures 
with  each  other  and  with  additional  predictors  will  help  to  establish  their  validity  and  potential 
usefulness  in  this  domain. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  LEADERSHIP:  Developing  leaders  in  a  changing  army  (WP  1 03) 

Accelerating  Leader  Development  (WP  269) 

•  Accelerating  Leader  Development  (WP  285) 

•  SELECT21:  Selection,  Classification  and  Performance  Metrics  for  the  Objective  Force 
(WP  257);  ATO:  IV.HS.  2002.01. 

Overview 

The  Army  (and  other  organizations)  must  have  leaders  or  potential  leaders  who 
continuously  pursue  the  development  of  leadership  skills  and  who  are  flexible  and  adaptable 
through  their  involvement  in  self-initiated  development  and  learning  experiences.  The  Army 
cannot  rely  solely  on  mandated  training  or  learning  experiences — it  needs  leaders  or  potential 
leaders  to  be  motivated  to  pursue  self-initiated  development  of  leadership  skills  and 
characteristics.  Total  development  must  be  motivated  and  initiated  from  within  the  person.  But 
little  is  known  about  predicting  leadership  development  behavior  because  most  research  has  been 
directed  at  predicting  the  performance  of  leaders.  Leadership  development  and  leader 
performance  are  not  the  same  thing.  Likewise,  motivation  to  be  a  leader  and  motivation  to 
develop  leadership  skills  are  not  the  same.  The  Army  needs  a  better  understanding  of  leadership 
development  and  those  behavioral  constructs  and  assessment  methods  that  are  relevant  to 
leadership  development,  especially  those  that  involve  individual  sustained  effort  to  develop 
leadership  attributes.  Basic  research  that  addresses  these  issues  can  ultimately  lead  to  applied 
research  which  enables  better  selection  and  development  of  leaders. 

A  key  first  step  in  this  process  is  to  develop  behavioral  variables/constructs  and 
assessment  methods,  including  a  network  of  related  variables,  that  would  help  establish  the 
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validity  and  relevance  of  these  new  variables  in  this  domain.  The  variables  will  be  explored 
within  a  system  of  related  constructs,  including  antecedents  and  consequences,  to  establish  the 
relevance  to  other  variables  within  the  leadership  development  domain.  There  will  be  two 
general  sets  of  variables  examined  in  this  research.  The  first  set  of  variables  includes  variables 
being  introduced  here  as  new  to  the  leadership  domain.  This  set  includes:  motivation  for 
leadership  development,  self-efficacy  (or  self-confidence)  for  leadership  development,  and 
beliefs  about  the  improvability  of  leadership  skills/attributes  (e.g.  beliefs  that  leadership  can  be 
developed  vs.  leadership  is  bom).  In  addition,  participation  in  leadership  development  activities 
for  the  purpose  of  leaming/developing  leadership  skills  will  also  be  examined.  The  second  set  of 
variables  to  be  examined  are  traditional  personality  and  ability  variables,  as  well  as  selected 
variables  from  the  employee  development  literature,  that  can  be  examined  as  predictors  of  these 
new  variables.  In  general,  it  is  expected  that  hypothesized  relationships  of  these  leadership 
development  variables  with  each  other  and  with  additional  predictors  will  help  to  establish  their 
validity  and  relevance  in  the  leadership  development  domain. 

Research  Approach 

A  two-wave  survey  method  will  be  used  to  collect  data  from  people.  The  source  of 
subjects  will  be  an  on-line  research  pool.  Registered  participants  (currently  N  >  46,000)  have  varied 
demographic  characteristics  including  racial  makeup,  gender,  education,  and  a  varied  list  of 
occupations.  Working  adults  of  a  wide  range  of  ages  will  be  included,  which  will  allow  the  study 
to  focus  on  both  younger  and  more  mature  adults. 

The  two  sets  of  variables  described  above  will  be  examined  in  the  following  way.  First, 
the  relationships  among  the  leadership  development  constructs  will  be  tested  when  data  for  the 
total  set  of  these  measures  is  collected  in  two  time  periods.  By  collecting  data  for  these  measures 
twice,  this  allows  for  a  retest  interval  to  examine  the  stability  of  the  constructs  across  time  and 
also  for  prospective  prediction  of  some  of  the  constructs  onto  others  at  a  later  time.  For  example, 
self-efficacy  for  leadership  development  at  time  1  can  be  used  to.  predict  motivation  to  develop 
leadership  skills  at  both  time  1  and  time  2.  Likewise,  motivation  to  develop  leadership  skills  can 
be  used  to  predict  not  only  participation  in  leadership  development  activities  at  time  1  but  also  at 
time  2.  So  the  relationships  among  the  leadership  constructs  can  be  examined  when  the  data  are 
all  collected  at  time  1 .  They  can  also  be  examined  when  the  data  are  collected  again  at  time  2 
and  the  stability  and  prediction  across  time  can  also  be  tested. 

The  second  set  of  variables  (the  traditional/existing  constructs)  will  also  be  examined  in 
relation  to  the  leadership  variables.  These  data  will  also  be  collected  at  time  1  and  examined  in 
relation  to  the  leadership  constructs  at  time  1,  and  also  the  leadership  constructs  that  are 
measured  again  at  time  2.  This  will  also  allow  prospective  prediction  by  some  of  the  traditional 
constructs  onto  others  at  a  later  time.  For  example,  personality  can  be  used  to  predict  self- 
efficacy  for  leadership  development  measured  at  both  time  1  and  at  time  2. 

Accomplishments  -  N/A-New  Start. 

Contributions  to  Basic  Science 

There  has  been  a  good  bit  of  research  on  predicting  leader  performance.  However,  there 
has  been  relatively  little  research  on  predicting  leadership  development  behavior. 
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For  example,  while  motivation  to  lead  and  self-efficacy  for  leadership  have  been  explored  in 
research,  self-efficacy  for  leadership  development  and  motivation  to  develop  leadership  skills 
have  not.  Also,  especially  given  differing  ideas  about  whether  leaders  are  bom  or  made,  there 
may  be  differences  in  how  aggressively  and  effectively  potential  leaders  pursue  the  development 
of  leadership  skills.  Successful  development  of  leadership  skills  depends  on  strong  motivation  to 
develop  these  skills.  The  current  research  will  test  a  model  of  leadership  development  motivation 
that  specifically  targets  motivation  to  develop  leadership  skills. 

A  recent  research  article  (Ruvolo,  Petersen  &  LeBouef,  2004,  Consulting  Psychology 
Journal )  authored  by  officers  in  the  U.S.  Army  is  titled  “Leaders  Are  Made,  Not  Bom,”  which 
strongly  reflects  the  notion  that  leadership  can  be  developed.  Further,  there  are  many  examples 
of  successful  leadership  training  and  development  programs.  However,  it  is  well  known  that 
people  differ  markedly  on  their  beliefs  that  leaders  are  ‘bom  or  made.’  Beliefs  that  leadership 
attributes  are  innate  or  fixed  and  are  not  acquirable  or  ‘improvable’  might  be  a  fundamental 
belief  that  can  set  the  stage  for  someone’s  overall  motivation  for  leadership  development.  To  the 
extent  that  a  person  believes  that  leaders  are  bom  and  not  made,  this  can  be  a  substantial 
impediment  to  optimal  motivation  for  developing  leadership  skills.  This  research  will  explore 
this  belief  and  the  role  that  the  construct  plays  in  leadership  development  motivation.  In 
addition,  the  similarities  and  differences  between  constructs  associated  with  leadership 
performance  and  leadership  development  will  be  addressed  conceptually  and  empirically.  This 
includes  the  distinction  between  self-efficacy  for  leadership  vs.  self-efficacy  for  leadership 
development,  and  motivation  to  lead  vs.  motivation  to  develop  leadership  skills.  Differences  in 
the  antecedents  and  outcomes  of  these  constructs  will  also  be  explored  to  establish  their  distinct 
construct  validity.  These  issues  have  not  been  explored  previously  in  relation  to  leadership 
development  but  do  have  great  theoretical  and  practical  promise. 

Potential  Army/Military  Applications 

Developing  these  new  constructs  in  the  leadership  domain  and  establishing  the  construct 
validity  of  them  in  basic  research  could  lead  to  applied  research  in  key  areas  addressed  by  ARI. 
Generally,  this  applied  research  could  fall  into  two  categories:  individual  difference  predictive 
studies  and  experimental  research  in  which  manipulations  are  used.  First,  these  constructs  and 
associated  assessment  methods,  along  with  traditional  predictors  such  as  intelligence  and 
personality,  could  be  used  as  predictors  of  performance,  motivation,  development  and 
adaptability  in  Army  training  and  field  settings.  To  the  extent  that  these  measures  predict 
effectiveness  in  adapting  to  leadership  roles  and  learning  and  developing  necessary  skills,  these 
tools  might  be  used  in  selection  and  placement  within  the  Army  to  identify  those  individuals  who 
will  likely  most  benefit  from  leadership  training  and  who  are  most  likely  to  take  initiative  to 
continually  develop  and  improve  leadership  relevant  characteristics  in  themselves.  These 
individual-difference  constructs  and  assessment  tools  might  be  explored  in  applied  research  for 
relationships  with  key  behavioral  and  performance  criteria  relevant  to  leadership  and  leadership 
development  and  learning  in  practical  Army  settings. 

There  is  also  a  second  way  that  developing  these  new  constructs  in  the  leadership 
domain  in  basic  research  could  lead  to  applied  research  in  key  areas  addressed  by  ARI.  Research 
might  be  done  on  ways  of  influencing  these  constructs  in  Army  personnel  to  improve  motivation 
for  leadership  development.  Variables  such  as  self-efficacy  are  subject  to  enhancement  through 
mastery  experiences,  persuasion,  modeling  or  vicarious  learning.  Self-efficacy  for  leadership 
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development  might  also  be  influenced  through  these  types  of  tactics.  For  example,  some 
research  has  manipulated  expectations  for  military  personnel  and  found  effects  for  increasing 
self-efficacy.  Applied  research  might  be  conducted  to  determine  whether  self-efficacy  for 
leadership  development  can  be  influenced,  and  whether  this  enhances  motivation  for 
development  of  leadership  skills  and  subsequent  development  and  performance.  Prior  research 
and  theory  suggests  that  this  should  be  possible. 

It  should  also  be  possible  to  conduct  applied  research  on  ways  in  which  beliefs  about 
improvability  of  leadership  skills  can  be  influenced.  Prior  research  in  settings  not  involving 
leadership  and  leadership  development  suggests  that  implicit  theories  of  ability  (beliefs  about 
improvability  of  attributes)  are  susceptible  to  influence.  It  should  be  possible  to  determine  if 
influences  on  beliefs  about  improvability  of  leadership  skills  have  a  subsequent  influence  on 
motivation  to  develop  leadership  skills  and  attributes  as  people  increasingly  believe  that  it  is 
possible  to  develop  and  improve  specific  characteristics.  Thus,  to  the  extent  that  these  constructs 
are  developed  and  validated  in  this  basic  research,  additional  applied  research  could  follow. 

Future  Plans 

In  the  coming  year  (year  1  of  a  3 -year  project),  the  literature  will  be  reviewed  to 
determine  the  content  of  the  newly  prepared  leadership  development  measures.  These  will  be 
compiled  based  on  that  review.  Pilot  or  pre-work  will  be  conducted  to  determine  the  length  of 
time  taken  to  complete  the  measures  and  whether  there  are  any  identifiable  problems  with  the 
format  and  wording.  A  computerized/web  version  of  the  measures  will  also  be  developed  this 
year. 
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Developing  Effective  Military  Leaders: 

Facilitating  the  Acquisition  of  Experience-Based,  Tacit  Knowledge 

Contract  #:  DASW01-00-K-0014  Contract  Dates:  9/01/00  -  9/30/04 

Institution:  Yale  University  PI:  Robert  J.  Sternberg 

CoPI:  Cynthia  T.  Matthew 
Liaison:  Joseph  Psotka,  LDRU 

Problems/  Research  Question(s)  -  Can  reflection  methods  based  on  Sternberg’s  theory  of 
practical  intelligence  enhance  practical  problem  solving  skills  and  the  ability  to  acquire 
experience-based  (tacit)  knowledge? 

Technical  Barrier(s)  -  How  we  learn  from  experience  and  acquire  experience-based  (tacit) 
knowledge  is  not  well  understood.  Prior  work  has  not  provided  models  and  tools  that  can  be  used 
to  measure  and  develop  practical  problem  solving. 

Significance/Impact  for  Basic  Research  -  This  project  will  extend  current  knowledge  about  the 
thought  processes  involved  in  how  we  learn  from  experience  and  develop  methods  to  facilitate  it 
for  applications  in  the  military,  particularly  leadership  development. 

Potential  Transitions  -  This  research  can  be  used  to  promote  leader  development  practices  in 
institutional  leader  development  in  the  Officer  Education  System  (OES)  and  the  Non¬ 
commissioned  Officer  Education  System  (NCOES).  This  objective  would  be  sustained  with 
technologies  to  move  battlefield  experiences  into  the  schoolhouse,  complemented  by  new 
technologies  for  self-development  opportunities,  and  improved  reachback  to  the  institutions  for 
better  leverage  of  operational  experience. 

Overview 

A  variety  of  reflection  methods  have  been  used  to  enhance  or  foster  the  exchange  of  tacit 
knowledge  in  a  range  of  workplace  settings.  However,  there  appears  to  be  little  research  that  has 
substantiated  the  effectiveness  of  these  methods.  Moreover,  if  reflection  does  indeed  enhance 
one’s  capacity  to  learn  effectively  from  experience,  it  is  unclear  what  types  of  reflection  methods 
may  be  most  effective.  These  questions  were  addressed  in  two  field  experiments,  which  tested 
the  effect  of  reflection  methods  based  on  Sternberg’s  theory  of  practical  intelligence,  which  is 
distinguished  from  analytic  and  creative  intelligence  in  this  innovative  theory  of  successful 
intelligence. 

Consistent  with  the  theory  of  practical  intelligence,  three  types  of  reflection  methods 
were  developed  that  draw  attention  to:  1)  factors  considered  in  problem  identification  (reflection 
on  condition);  2)  chosen  course  of  action  and  action  outcomes  (reflection  on  action);  and  3)  a 
combination  of  both  reflection  on  condition  and  action.  To  measure  the  differential  effect  of 
these  interventions  on  practical  problem  solving,  two  new  military-specific  case  study 
instruments  were  developed  to  assess  practical  skills  and  tacit  knowledge  in  military  leadership. 

It  was  predicted  that  reflection  methods  that  focus  reflection  on  both  the  condition  and 
action  aspects  of  practical  problem  solving  would  improve  scores  on  tacit  knowledge  measures 
more  than  methods  that  focus  on  one  or  the  other  aspect  exclusively.  It  was  also  expected  that 
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reflection  methods  that  focus  reflection  on  both  the  condition  and  action  aspect  of  practical 
problem  solving  would  be  superior  to  traditional  analytical  methods  of  reflection  (e.g.,  assessing 
the  advantages  and  disadvantages  of  a  particular  action  in  a  given  situation). 

Research  Approach 

Two  related  experimental  field  studies  were  conducted  to  investigate  the  effect  of  theory- 
based  reflection  methods  on  practical  problem  solving  in  Army  officer  and  college  student 
samples  using  measures  to  assess  experience-based  (tacit)  knowledge.  The  first  study  involving 
an  Army  officer  sample  was  discontinued  midway  through  data  collection  because  access  to 
officers  was  suspended  due  to  the  Iraq  War.  The  second  study  was  designed  to  complete  the 
testing  of  hypotheses  with  a  college  student  sample. 

Two  types  of  situational  judgment  tasks  were  used  to  assess  experience-based  tacit 
knowledge.  The  first  type  of  measure  contained  brief  problem  situations  (vignettes)  followed  by 
response  options  that  participants  evaluate.  The  second  type  of  measure,  developed  for  this 
investigation,  contained  extended  problem  situations  (scenarios)  followed  by  open-ended 
questions  to  which  participants  construct  a  response.  Three  reflection  methods  (reflection  on 
condition,  reflection  on  action,  and  combined  reflection  on  condition  and  action)  based  on  the 
theory  of  practical  intelligence  were  designed  and  tested.  In  a  single  three-hour  session, 
participants  were  administered  tacit  knowledge  pre-test  assessments,  an  experimental  reflection 
method  or  control  condition  (practice  or  traditional  analytic  reflection  method),  and  tacit 
knowledge  post-test  assessments.  Statistical  analyses  were  conducted  to  test  the  effect  of  each 
experimental  reflection  method  on  tacit  knowledge  post-test  performance. 

Accomplishments 

Findings  provide  preliminary  evidence  that  tacit  knowledge  can  be  facilitated  by 
reflection  methods  that  are  derived  from  the  theory  of  practical  intelligence.  The  relative 
effectiveness  of  reflection  method  seemed  to  depend  on  the  task  employed. 

For  tacit  knowledge  vignettes,  which  provide  limited  information  about  a  specific 
problem  situation  and  ask  participants  to  assess  response  strategies,  the  combined  reflection 
method  was  effective  in  improving  tacit  knowledge  scores.  Scores  on  vignettes  are  measures  of 
distance  from  expert  responses;  lower  scores  reflect  higher  levels  of  tacit  knowledge.  Figure  1 
illustrates  scores  on  pre-  and  posttest  vignettes  for  each  type  of  reflection  method.  As  can  be  seen 
in  the  figure,  the  combined  method  showed  the  greatest  reduction  in  distance  from  the  experts  in 
the  post-test. 

Figure  2  displays  tacit  knowledge  scores  on  scenarios  in  pre-  and  post-tests  for  each  of 
the  reflection  methods.  Scenarios  provide  complex  information  about  a  problem  situation  as  it 
unfolds  over  time  and  asks  participants  to  construct  a  response  to  questions  designed  to  measure 
tacit  knowledge.  For  scenarios  higher  scores  reflect  higher  levels  of  tacit  knowledge.  As  can  be 
seen  in  the  figure,  the  reflection  on  action  method  was  the  most  effective  in  improving  scenario 
scores  followed  by  the  reflection  on  condition  method.  Perhaps  because  of  all  of  the  information 
that  must  be  managed  in  the  scenario  task,  it  may  have  been  more  difficult  for  participants  to 
benefit  from  the  combined  approach  in  a  brief  intervention. 
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Figure  1 .  Performance  on  vignettes  in  pre-  (1)  and  post-test  (2) 
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Figure  2.  Performance  on  scenarios  in  pre-  (1 )  and  post-test  (2) 


Reflection  Methods 

- control 

- condi 

. comb 

— - action 


52 


Contributions  to  Basic  Research 


This  investigation  illuminates  a  previously  unexplored  aspect  of  Sternberg’s  theory  of 
intelligence  by  outlining  the  cognitive  processes  involved  in  acquiring  tacit  knowledge  from 
experience,  the  problem  solving  skills  involved  in  solving  complex  practical  problems,  and  the 
interaction  between  knowledge  and  problem  solving  in  expert  performance.  The  reflection 
methods  and  case  study  assessments  developed  for  this  investigation  can  be  used  for  further 
research  in  this  area  of  inquiry. 

Potential  Army/Military  Applications 

Officers  of  all  four  ranks  who  participated  in  the  study  indicated  that  they  learned  from 
the  reflection  methods,  the  methods  were  useful  for  understanding  how  to  learn  from  experience, 
relevant  to  the  leadership  problems  they  typically  face,  and  as  a  result  of  participation,  they  plan 
to  use  future  experiences  as  opportunities  for  learning. 

Reflection  methods  developed  this  project  can  be  directly  applied  to  leadership 
development  in  the  Army  at  multiple  levels  of  leadership.  A  manual  of  materials  was  created  for 
this  purpose.  The  methods  featured  in  the  manual  can  also  be  adapted  to  specific  training 
situations. 

Final  Summary 

This  investigation  showed  that  even  very  brief  reflection  methods  based  on  the  theory  of 
practical  intelligence  can  improve  practical  problem  solving.  Reflection  methods  that  were 
developed  based  on  the  theory  of  practical  intelligence  effectively  promoted  practical  problem 
solving  in  study  participants.  These  reflection  methods  appear  to  be  more  effective  than  no  reflection 
or  traditional,  analytic  approaches  to  reflection  on  experience.  However,  all  reflection  methods  are 
not  alike.  The  effectiveness  of  a  particular  reflection  method  seemed  to  depend  upon  the  complexity 
of  the  task.  Time  may  determine  the  extent  to  which  a  particular  method  is  effectiveness  given  a 
particular  type  of  task. 

While  a  variety  of  reflection  methods  have  been  advocated  and  used  to  enhance  or  foster  the 
exchange  of  tacit  knowledge  in  a  range  of  workplace  settings,  the  effectiveness  of  these  methods 
may  vary  considerably  depending  upon  how  they  are  designed  and  implemented.  This  investigation 
is  a  primary  step  in  identifying  theory-based  methods  that  facilitate  the  development  and  use  of  tacit 
knowledge.  There  is  much  more  work  to  be  done  to  verify  the  effect  of  reflection  methods  on 
learning  and  performance,  identify  that  factors  that  moderate  the  relationship  between  reflection 
method  and  effectiveness,  and  examine  the  transfer  of  training  over  time.  Further  research  has  the 
potential  to  uncover  more  about  the  relationship  between  reflection  method  type,  time,  task  type,  and 
other  factors  that  may  have  bearing  on  practical  problem  solving  and  the  acquisition  of  tacit 
knowledge.  This  knowledge  can  have  direct  effect  on  specific  types  of  training  and  development  in 
the  Army  and  in  other  organizations.  For  example,  in  the  Army  the  differential  effectiveness  of 
reflection  method  and  modes  of  implementation  may  depend  on  level  of  leadership.  More 
experienced  officers  whose  experience  may  no  longer  be  appropriate  in  the  current  environment, 
may  require  different  methods  than  less  experienced  officers  who  are  just  beginning  to  build  a 
network  of  tacit  knowledge  in  military  leadership. 
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Identifying  Individual  Attributes  and  Learning  Principles  that  Foster  Adaptive 
Performance  and  Promote  Rapid  Adaptability  Skill  Acquisition  through 

Multiple  Modes  of  Development 

Contract  #:  Not  yet  determined.  Contract  Dates:  1/15/05-7/31/07 

Institution:  George  Mason  University  PI:  Stephen  J.  Zaccaro 

Liaison:  Joseph  Psotka,  LDRU 

Problem/Research  Questions  - 

•  What  combinations  of  leader  and  team  processes  and  attributes  lead  to  maximum 
adaptive  performance  in  today’s  military  operational  environments? 

•  What  training  and  development  principles  produce  the  most  effective  acquisition  and 
growth  of  leader  adaptability  skills? 

•  What  individual  attributes  moderate  the  impact  of  training  and  development  interventions 
on  adaptability  skills? 

Technical  Barriers  -  Past  theories  and  models  of  leadership  and  team  training  are  inadequate 
for  understanding  how  to  grow  adaptive  leaders  and  teams.  Training  and  developing  adaptability 
skills  for  both  individual  soldiers  and  teams  requires  training  strategies  that  are  different  from 
those  traditionally  employed  by  the  Army. 

Significance  /Impact  for  Basic  Research  -  The  successful  completion  of  this  research  effort 
should  contribute  significantly  to  the  literature  on  leader  and  team  development,  in  particular  on 
strategies  for  growing  individual  and  team  level  adaptability  skills  through  formal  instruction, 
developmental  work  assignments,  and  self-development.  We  should  also  add  new  information  to 
the  small  but  growing  literature  on  leader  attributes  patterns  and  their  influence  on  leader 
adaptive  performance. 

Potential  Transitions  -  The  products  of  this  research  should  provide  the  basis  for  applied 
research  through  such  efforts  as  DoD  and  Army  SBIRs  and  STTRs.  Specific  topics  would 
include: 

•  The  development  of  formal  training  programs  that  target  leader  adaptability  skills. 

•  The  refinement  and  validation  of  assessment  tools  that  can  be  used  in  Army  training  and 
development  activities  to  measure  developmental  work  experiences,  learning  attributes, 
and  adaptive  performance  processes. 

Overview 

The  overall  purposes  of  this  research  effort  are  to  expand  current  understanding  about  the 
nature  of  adaptive  leadership  and  to  explore  the  most  effective  ways  of  developing  adaptability 
skills  through  three  modes  of  Army  training  -  formal  instruction,  operational  assignments,  and 
self-development.  We  are  examining  several  research  questions  related  to  these  purposes, 
including  how  the  operations  of  leadership  change  as  military  environment  conditions  become 
more  dynamic  and  adversarial.  We  will  also  explore  the  leader  attributes  that  promote 
effectiveness  under  such  environmental  conditions.  The  specification  of  adaptive  leader 
processes  and  attributes  sets  the  stage  for  a  consideration  of  best  practices  in  leader  training  and 
development  to  foster  growth  in  leader  adaptability.  Accordingly,  we  will  investigate  training 
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and  development  principles  that  are  likely  to  produce  the  most  effective  growth  of  particular 
leader  adaptability  skills.  We  will  also  examine  how  these  principles  vary  across  different 
modes  of  Army  leader  development. 

Research  Approach 

Our  research  approach  consists  of  a  series  of  studies  in  both  laboratory  and  field  settings. 
We  expect  to  complete  2  laboratory  studies  with  undergraduate  students  in  which  we  will  vary 
the  content  of  training  and  the  means  by  which  training  can  be  presented  to  future  leaders. 
Specifically,  we  will  focus  on  teaching  adaptive  thinking  skills  and,  in  the  course  of  training, 
employ  a  variety  of  different  performance  scenarios  to  practice  the  use  of  these  skills.  In  these 
studies  we  will  also  focus  on  student  learning  strategies  and  on  specific  training  design 
approaches  that  recent  literature  suggest  should  foster  adaptability  skills. 

We  also  expect  to  complete  3  studies  in  several  classroom  settings.  These  three  studies 
will  use  primarily  survey  methods  and  leader  simulation  exercises.  The  surveys  will  examine  the 
operational  and  developmental  assignments  completed  by  organizational  managers  and  military 
officers,  with  the  intent  of  identifying  certain  aspects  of  these  assignments  that  are  most  likely  to 
foster  gains  in  adaptability  skills.  The  exercises  will  be  used  to  link  growth  in  adaptability  skills 
through  self-development  and  developmental  assignments  to  displayed  adaptability  in  leadership 
simulations.  We  expect  to  complete  these  exercises  with  undergraduate  students,  MBA  students, 
and  military  personnel. 

Accomplishments 

N/A  -  New  Start 
Contributions  to  Basic  Science 

This  research  effort  should  provide  substantial  benefits  by  enhancing  our  understanding 
of  how  to  train  and  develop  adaptability  skills  through  formal  course  instruction,  as  well  as 
through  officer  assignments  and  self-development  efforts.  This  research  effort  should  also 
contribute  to  the  growing  literature  on  how  different  characteristics  of  leaders  are  linked  in 
patterns  to  influence  leader  adaptive  performance,  as  well  as  the  leader’s  responsiveness  to 
different  training  interventions. 

Potential  Army/Military  Applications 

Some  of  the  anticipated  products  of  this  research  effort  include  training-based  assessment 
tools  and  training  protocols  that  can  contribute  to  more  effective  development  of  soldiers  and 
offices.  Several  training  units,  including  those  at  Ft.  Benning  and  Ft.  Leavenworth,  are 
particularly  interested  in  developing  more  adaptive  leaders.  We  expect  the  products  of  our 
research  to  support  these  and  similar  efforts. 
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Future  Plans 


The  formal  start  date  for  this  effort  is  January  1,  2005.  We  expect  in  the  first  year  to: 

•  Design  the  training  manipulations  for  the  laboratory  studies 

•  Complete  data  collection  for  the  first  of  two  laboratory  studies 

•  Refine  and  extend  existing  surveys  that  measure  the  quality  of  leader  developmental 
assignments. 

•  Administer  the  revised  draft  of  the  developmental  assignment  survey  to  managers  and 
military  personnel. 

•  Complete  a  research  report  on  the  conceptual  foundations  of  this  research  effort. 
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RACO  RESEARCH  OBJECTIVE  #3: 

Provide  fundamental  knowledge  for  identifying,  assessing,  and 
assigning  quality  personnel  for  the  evolving  Army. 

Research  in  this  section  is  directed  toward  identifying  and  measuring 
the  aptitudes  and  skills  that  are  unique  to  the  human  performance 
requirements  of  the  Future  Force.  Exploring  the  sociological  and 
psychological  factors  that  could  influence  recruitment,  retention,  and 
Army  performance  are  important  aspects  of  this  research  objective. 
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An  Interactionalist  Analysis  of  Soldier  Retention 
Across  Career  Stages  and  Time 

Contract  #:  W74V8H-04-K-0002  Contract  Dates:  6/1/04-9/30/05 

Institution:  Texas  A&M  University  PI:  Gilad  Chen 

Co-PI:  Robert  Ployhart 
Liaison:  Michael  Rumsey,  SARU 

Problem(s)/Research  Question(s)  -  How  do  personal,  situational,  and  temporal  factors  help 
explain  retention  processes  at  different  career  stages? 

Technical  Barrier(s)  -  Models  of  retention  have  yet  to  consider  dynamic  processes  leading  to 
turnover,  and  whether  retention  processes  might  differ  at  different  career  stages 

Significance/Impact  for  Basic  Research  -  Understanding  how  and  why  the  decision  to  stay  or 
leave  an  organization  unfolds  over  time,  and  whether  personal  and  situational  factors 
differentially  drive  these  processes  at  different  career  stages. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  RETAIN:  Strategies  to  Retain  Quality  Soldiers  (WP  271) 

•  Soldier  Attitudes  and  Opinions  in  a  Changing  Army  (WP  1 02) 

•  SELECT21:  Selection,  Classification,  and  Performance  Metrics  for  the  Objective  Force 
Soldier  (WP  257)  and  ATO:  IV.HS.2002.01 . 

Overview 

As  a  result  of  both  the  collapse  of  the  former  Soviet  Union  and  the  horrendous  events  of 
September  11,  2001,  the  U.S.  has  engaged  in  a  series  of  conflicts  around  the  globe,  oftentimes 
simultaneously.  In  today’s  reality,  and  probably  even  more  so  in  the  future,  the  U.S.  armed 
forces  must  be  flexible  and  adaptable  in  their  ability  to  deploy  the  same  troops  to  qualitatively 
different  types  of  conflicts,  with  vastly  different  purposes  (e.g.,  dismantling  terror  and/or  drug 
networks,  homeland  defense,  peace  keeping)  and  in  different  geographical  locations. 
Consequently,  the  U.S.  armed  forces  have  put  a  high  premium  on  selecting,  training,  and 
maintaining  the  sufficient  human  resources  required  for  being  competitive  in  this  new  global 
reality.  Furthermore,  effective  retention  has  never  been  as  consequential  as  it  is  today,  given  the 
high  costs  involved  in  attracting,  selecting,  and  training  soldiers. 

Unfortunately,  current  theoretical  models  of  retention  provide  an  insufficient  basis  for  our 
understanding  of  retention  in  such  complex  and  dynamic  work  environments.  The  present 
research  addresses  three  limitations  of  previous  retention  research: 

1.  Organizations  can  enhance  retention  through  management  practices  directed  at 
creating  more  positive  work  situations,  and  through  staffing  practices  directed  at 
selecting  employees  who  would  better  fit  the  work  environment.  Accordingly,  the 
first  contribution  of  the  present  research  is  the  simultaneous  examination  of  both 
person  variables  (cognitive  ability)  and  situational  variables  (social  support  and  work 
challenge)  as  predictors  of  the  retention  process.  In  particular,  we  propose  that  lower 
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levels  of  ability  and  higher  levels  of  social  support  and  work  challenge  are  associated 
with  more  positive  levels  of  job  attitudes,  and  thus  lower  likelihood  of  turnover.  We 
further  consider  possible  person-by-situation  interactions,  and  propose  that  the 
relationships  between  ability  and  job  attitudes  are  less  negative  when  social  support 
and  work  challenge  are  higher. 

2.  There  is  now  a  growing  awareness  that  factors  which  affect  work  experiences  may 
vary  across  career  stages,  given  employees  require  and  desire  different  things  at 
different  stages  of  their  careers.  In  the  context  of  the  U.S.  Army,  for  instance,  there 
could  be  different  reasons  why  first  and  second  term  soldiers  decide  to  re-enlist.  At 
early  career  stages,  soldiers  are  adjusting  to  their  new  military  career,  and  thus  social 
support  is  particularly  critical.  However,  at  later  career  stages,  soldiers  are  more 
likely  to  appreciate  and  benefit  from  challenging  work  assignments.  Accordingly,  a 
second  contribution  of  our  research  is  the  examination  of  whether  retention  processes 
are  affected  by  different  factors  at  different  career  stages.  In  particular,  we  propose 
that  cognitive  ability  and  social  support  are  more  likely  to  relate  to  job  attitudes  and 
retention  at  earlier  career  stages,  whereas  work  challenge  is  more  likely  to  relate  to 
job  attitudes  and  retention  at  later  career  stages. 

3.  Finally,  given  that  employees  in  many  organizations  (including  the  U.S.  Army) 
transition  from  one  work  environment  or  project  to  another  quite  frequently,  it  is 
critical  to  know  whether  such  transitions  are  associated  with  changes  in  retention 
decisions  as  well  as  in  proximal  drivers  of  retention  outcomes  (e.g.,  job  attitudes),  and 
whether  capturing  particular  patterns  of  change  (i.e.,  growth/increase  or 
decline/decrease)  in  variables  capturing  the  retention  process  can  help  explain 
additional  variance  in  turnover.  In  our  research,  we  propose  that  more  positive 
changes  in  job  attitudes  are  associated  with  decreased  likelihood  of  turnover. 
Moreover,  we  argue  that  changes  in  job  attitudes  capture  a  unique  phenomenon 
different  from  average  levels  of  job  attitudes,  and  thus  changes  in  job  attitudes  should 
account  for  additional  variance  in  retention  decisions  over  and  above  average  levels 
of  job  attitudes. 

Research  Approach 

We  are  currently  in  the  process  of  testing  our  propositions  using  field  survey  data 
collected  by  Walter  Reed  researchers  (called  the  OPTEMPO  project)  from  over  one  thousand 
soldiers  at  multiple  times  over  the  course  of  two  years.  The  survey  data  were  also  linked  to 
soldiers’  ability  scores,  which  were  collected  during  the  selection  process  into  the  U.S.  Army. 
The  surveys  captured  multiple  aspects  of  the  work  environment,  including  social  support  (i.e., 
perceived  quality  of  officer  and  non-commisioned  officer  leadership,  co-worker  support)  and 
work  challenge  (i.e.,  job  challenge,  significance  of  work  tasks,  and  whether  soldiers  were  in 
deployement,  in  training,  or  in  their  garrison  base).  At  each  time  period,  soldiers  also  indicated 
their  job  attitudes  (i.e.,  job  satisfaction,  job  involvement,  and  general  morale)  and  intention  to  re¬ 
enlist  with  the  Army.  Finally,  actual  turnover  and  career  stage  data  are  also  available.  These 
rich  data  are  unique  in  that  they  allow  us  to  capture  and  explain  retention  processes  as  they 
unfold  over  time,  and  at  different  career  stages. 
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Accomplishments 

N/A  -  New  Start 

Contributions  to  Basic  Research 

This  project  contributes  to  the  employee  retention  literature  in  several  important  ways. 
First,  by  integrating  person  and  situation  theories  of  employees  retention,  this  research  provides  a 
more  comprehensive  picture  of  the  retention  process  and  its  determinants.  Second,  this  research 
highlights  the  potential  differences  in  drivers  of  retention  across  career  stages.  This  is  important, 
given  it  highlights  key  differences  in  employee  motivation  and  retention  at  different  career 
stages,  and  thus  refines  the  boundary  conditions  for  generalizing  employee  retention  theories. 
Finally,  moving  beyond  the  mostly  static  retention  research,  this  research  provides  a  richer,  more 
dynamic  view  of  how  and  why  employees’  decisions  to  stay  or  leave  the  organization  evolve 
over  time  vis-a-vis  their  experiences  at  work. 

Potential  Army/Military  Applications 

This  research  has  the  potential  to  advance  Army/military  applications  in  several  ways. 
First,  given  that  U.S.  armed  forces’  selection  systems  rely  heavily  on  cognitive  ability,  it  is 
important  to  ensure  that  more  qualified  soldiers  (i.e.,  those  higher  on  ability)  are  those  remaining 
in  the  military.  By  linking  cognitive  ability  and  the  retention  process,  and  by  examining  possible 
situational  moderators  of  this  relationship  (i.e.,  social  support  and  work  challenge),  the  present 
research  will  identify  plausible  means  of  retaining  more  able  soldiers.  Second,  identifying 
different  reasons  for  turnover  at  different  career  stages  could  be  translated  into  more  effective 
practices  directed  at  retaining  soldiers  at  different  career  stages.  In  particular,  if  our  hypotheses 
are  corroborated,  the  implication  would  be  to  emphasize  different  management  principles  to 
retain  soldiers  at  different  career  stages  -  i.e.,  social  support  would  particularly  important  when 
managing  first-term  soldiers,  and  increasing  work  challenges  would  be  critical  later  on  in 
soldiers’  careers.  Finally,  given  this  study  tracks  soldiers  over  the  course  of  2  years  as  they 
transition  across  different  work  assignments  (i.e.,  training  or  deployment),  it  can  inform  the 
military  regarding  soldiers’  experiences  resulting  from  these  transitions.  More  important,  this 
study  will  identify  various  factors  that  could  attenuate  or  accentuate  the  negative  or  positive 
experiences  of  soldiers’  work  transitions,  which  can  be  used  to  inform  programs  directed  at 
enhancing  leadership  effectiveness  and  retention.  Altogether,  this  research  will  likely  help  the 
military  improve  various  management  programs  directed  at  enhancing  retention,  and  help  align 
these  programs  with  the  military’s  staffing  strategies. 

Future  Plans 

Our  specific  plans  for  this  year  are  the  following.  First,  we  recently  finished  merging  the 
OPTEMPO  data  with  soldiers’  ability  and  attrition  data.  Second,  we’re  now  conducting 
numerous  psychometric  tests  to  confirm  the  validity  and  reliability  of  the  various  study 
measures.  Next,  we  will  conduct  various  analyses  to  examine  our  theoretical  propositions  and 
hypotheses. 
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Measuring  Average  Speed  of  Numerical  Reasoning 

Contract#:  143 5-04-03 -CT-74083  Contract  Dates:  10/1/03-9/30/05 

Institution:  University  of  Minnesota,  Twin  Cities  PI:  Mark  L.  Davison,  Ph.D. 

Co-PI:  Robert  Semmes 
Liaison:  Michael  Rumsey,  SARU 

ProbIem(s)/Research  Question(s):  Is  time-constrained  performance  within  a  given  cognitive 
task  domain  determined  by  one  ability,  called  level  of  performance,  or  by  at  least  two  abilities, 
level  of  performance  and  speed  of  performance? 

Technical  Barriers:  (a)  Undeveloped  theory  of  the  cognitive  abilities  responsible  for  time- 
constrained  performance  of  tasks  having  non-trivial  difficulty;  and  (b)  fallacious  models  and 
methods  for  measuring  speed  of  task  performance  on  cognitive  tasks  having  non-trivial 
difficulty. 

Significance/Impact  for  Basic  Research:  If  our  project  is  successful,  we  will  provide  the  first 
psychometrically  valid  approach  to  measuring  speed  of  performance  of  cognitive  tasks  having 
non-trivial  difficulty. 

Potential  Transitions 

This  research  could  potentially  lead  to: 

•  Improved  measures  of  critical  attributes  needed  by  soldiers  for  effective  performance  in 
assignments  for  which  mental  quickness  is  important;  enhanced  predictive  validity  of  the 
AFQT  if  performance  speeds  in  the  various  cognitive  domains  were  measured 

•  Measures  that  can  be  used  in  a  person-job  match  model  (i.e.,  methods  for  classifying  and 
assigning  force  members) 

•  Improved  training  programs  in  areas  where  mental  quickness  is  important. 

Overview 

Few  would  doubt  that  if  each  person  in  a  group  were  assigned  a  set  of  non-trivial  mental 
tasks  of  like  kind,  the  group’s  members  would  vary  in  how  successfully  and  rapidly  they 
performed  those  tasks.  Psychologists  have  known  for  several  decades  how  to  validly  measure  a 
person’s  ability  to  successfully  perform  tasks  in  a  delimited  cognitive  domain  when  no  time¬ 
limits  for  completing  the  tasks  are  imposed.  Surprisingly,  however,  they  have  yet  to  find  a  way 
to  validly  measure  a  person’s  overall  quickness  in  successfully  performing  tasks  randomly 
sampled  from  the  domain. 

One  of  the  most  important  achievements  in  the  field  of  psychological  measurement 
during  the  second-half  of  the  20th  century  was  the  development  of  models  to  measure  the  ability 
that  accounts  for  a  person’s  success  in  performing  tasks  sampled  from  a  given  cognitive  task 
domain,  irrespective  of  the  particular  tasks  sampled.  The  ability  these  models  measure  is  called 
level  of  performance,  or  simply  level.  Such  models,  however,  depend  on  the  strong— and  often 
overlooked  or  ignored — assumption  that  task  performance  is  self-paced.  It  is  natural  to  ask 
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whether  the  same  ability,  level  of  performance,  also  accounts  for  success  when  task  performance 
is  constrained  by  time-limits. 

One  possible  answer  to  this  question  is  that  a  second  ability,  speed  of  performance, 
comes  into  play  when  the  conditions  of  task  performance  include  time-limits.  Since  early  in  the 
20th  century  psychologists  interested  in  individual  differences  have  speculated  that  a  general 
speed  ability  influences  performance  on  timed  tests  of  general  intelligence.  All  efforts  to 
measure  such  an  ability,  however,  were  unsuccessful,  as  were  efforts  to  measure  speed  of 
performance  in  more  narrowly-defined  cognitive  task  domains.  Many  of  the  failed  attempts  to 
measure  a  speed  ability  were  due  to  researchers’  failure  to  recognize  that  any  meaningful  speed 
ability  that  might  exist  must  be  based  on  successful  task  performances  rather  than  a  combination 
of  successful  and  unsuccessful  performances. 

Another  fundamental  reason  why  no  one  was  able  to  measure  any  speed  ability  was  that 
investigators  failed  to  formulate  a  suitable  model  for  measuring  a  speed  ability  distinct  from 
level  ability.  Using  mean  or  median  correct  response  time  as  one’s  measure  of  speed  ignores  the 
likely  influence  of  item  difficulty  on  item  response  time.  A  person’s  median  correct  response 
time  on  a  random  sample  of  tasks  from  a  given  domain  will  vary  depending  on  the  particular  set 
of  tasks  the  person  performs.  The  dominant  measurement  model  for  most  of  the  20th  century, 
classical  test  theory’s  so-called  true-and-error  score  model,  is  also  unsatisfactory  for  measuring 
cognitive  speed.  That  model  postulates  that  success  on  a  test  item  (cognitive  task)  is  determined 
by  just  one  ability,  a  person’s  true  score  in  the  given  task  domain.  Among  its  serious 
shortcomings,  the  classical  test  theory  measurement  model  fails  to  recognize  the  separate 
influences  that  the  test-taker  and  the  test  item  have  on  performance  outcomes.  In  effect,  the 
model  tacitly  assumes  that  all  items  on  a  cognitive  test  have  the  same  characteristics. 

To  investigate  whether  one  or  two  abilities  account  for  task  performance  outcomes  when 
performance  is  constrained  by  time,  one  must  first  propose  a  probabilistic  model  of  task 
performance  that  recognizes  the  separate  influences  of  the  tesMaker  and  the  task.  Modem 
models  for  measuring  level  of  performance  do  recognize  those  separate  influences,  and  we 
ourselves  have  proposed  a  probabilistic  measurement  model  that  postulates  two  abilities  (two 
person  parameters)  and  three  item  characteristics  (three  item  parameters)  to  account  for 
successful  performance  on  a  given  item  when  the  item  has  an  assigned  time-limit .  Secondly, 
using  item  performance  outcome  data  (i.e.,  item  scores)  obtained  from  a  sample  of  subjects 
performing  under  time-constrained  conditions  (i.e.,  item  time-limits),  one  must  compare  the  fit  of 
a  model  postulating  level  of  performance  as  the  sole  ability  determining  performance  on  an  item, 
with  the  fit  of  a  model  postulating  two  abilities — level  and  speed — as  determining  performance. 
Indeed,  this  is  the  approach  we  have  adopted,  though  the  speed  ability  we  are  trying  to  measure 
is  an  average  speed  obtained  by  varying  across  items  the  time-limits  each  of  our  subjects 
encounters. 

Research  Approach 

We  are  conducting  a  field  study  with  college  students  in  psychology  courses  at  the 
University  of  Minnesota  as  the  research  subjects.  Students  who  participate  in  the  study  are  paid  a 
small  honorarium  and  given  extra  credit  points  in  their  psychology  course  in  return  for 
completing  both  phases  of  the  study.  We  began  by  defining  the  conditions  of  testing,  which, 
together,  constitute  the  power  performance  conditions  of  the  cognitive  task.  These  include 
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imposing  no  task-level  or  test-level  time  constraints;  instructions  urging  subjects  to  work  at  their 
own  pace  so  as  to  complete  as  many  tasks  (items)  correctly  as  they  can;  assessing  subjects  during 
the  time  when  they  are  most  awake;  making  sure  subjects  are  not  sick,  hungry,  or  taking  any 
performance-enhancing  drugs  other  than  possible  caffeine  or  nicotine;  assuring  that  the  testing 
environment  is  physically  comfortable  and  free  of  distractions;  and  making  sure  that  the  set  of 
tasks  the  subject  is  asked  to  perform  does  not  include  any  that  s/he  has  attempted  to  perform 
within  the  last  12  months. 

Approximately  600  subjects  will  be  assessed  in  each  of  two  Phases.  In  each  Phase 
subjects  take  one  or  the  other  of  two  equivalent  forms  of  a  numerical  reasoning  test  consisting  of 
items  drawn  from  the  Psychological  Corporation’s  Differential  Ability  Tests,  published  editions 
of  the  SAT  I,  and  published  editions  of  the  Graduate  Record  Exam’s  General  Test.  In  Phase  1 
testing,  we  assess  each  subject’s  level  ability  by  randomly  assigning  them  one  of  the  two 
alternate  forms  and  administering  it  under  self-paced  performance  conditions.  In  Phase  2  testing, 
each  subject  takes  the  alternate  test  form,  wherein  each  item  will  be  administered  with  a  response 
time-limit. 

Two  statistical  models  will  then  be  fit  to  the  Phase  2  testing  data.  One  model  assumes 
that  a  subject’s  level  of  performance,  estimated  using  the  Phase  1  data,  is  the  only  cognitive 
ability  determining  performance;  the  alternative  model  assumes  that  level  and  speed  jointly 
determine  Phase  2  performance.  The  Phase  2  data  will  be  analyzed  to  see  which  model  provides 
the  better  fit.  If  the  analysis  supports  the  hypothesis  that  a  speed  ability  in  part  accounts  for 
time-constrained  performance  of  numerical  reasoning  items,  then  it  would  appear  likely  that  a 
speed  ability  could  also  be  measured  in  one  or  more  of  the  task  domains  assessed  by  the  Armed 
Forces  Qualification  Test  (AFQT).  Measuring  such  speed  abilities  could  improve  the  AFQT’s 
predictive  validity  for  assigning  people  to  various  jobs  in  which  mental  quickness  is  important. 
The  test  would  most  likely  require  examinees  to  take  similar  items  under  self-paced  and  timed 
conditions  (with  carefully  chosen  item  time  limits)  to  obtain  separate  measures  of  performance 
level  and  performance  speed. 

Accomplishments: 

•  Replaced  most  of  the  original  item  pool  with  quantitative  reasoning  items  taken  primarily 
from  old,  published  editions  of  the  SAT-I  and  GRE  General  Test.  This  significantly 
improved  the  item-construct  match,  which  in  turn  strengthened  the  validity  and 
reproducibility  of  our  forthcoming  results. 

•  Overhauled  the  Psychology  Department’s  LabVIEW  program  to  improve  its  vocal 
recording  capacity  (in  initial  testing  in  the  Fall  of  2003,  the  program  failed  to  properly 
record  the  audio  files  of  nearly  50%  of  the  first  41  subjects).  The  new  program,  tested 
after  five  revisions,  performed  flawlessly  in  a  series  of  mock  sessions.  On  March  1,  2004, 
we  resumed  self-paced  testing  of  subjects. 

•  Administered  Phase  1  testing  to  307  subjects. 
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•  Commenced  scoring  Phase  1  tests  (includes  transcribing  the  WAV  file  recordings  to  a 
spreadsheet  file,  classifying  answers  as  correct,  incorrect,  or  not  attempted  if  a  subject 
prematurely  terminated  the  test  session). 

•  Completed  Stage  I  of  the  project  (i.e.,  collected  enough  Phase  1  test  data  to  allow  rational 
assignment  of  item-level  time  limits  for  each  of  the  items  that  will  be  administered  in 
Phase  2’s  time-constrained  testing). 

As  described  in  our  3rd  Quarter  Progress  Report,  it  became  evident  that,  in  order  to  assign  time 
limits  for  each  item  that  would  be  used  in  Phase  2,  we  needed  to  collect  self-paced  test  data  from 
150  students  rather  than  from  75,  as  originally  planned.  By  the  end  of  the  3rd  quarter,  we  still 
needed  Phase  1  data  from  15  students  to  reach  the  needed  300  subjects,  and  more  students  were 
recruited  over  the  summer.  Additionally,  it  became  clear  during  the  3rd  quarter  that  we  would 
need  more  time  to  collect  data  from  our  target  sample  of  600  students.  Given  the  dual  constraints 
of  competition  with  other  research  studies  for  subjects  in  the  Psychology  Department’s  subject 
pool,  and  of  competition  with  other  studies  and  classes  for  time  in  the  Department’s  computer- 
based  testing  laboratory,  we  realized  that  we  would  need  another  two  semesters  (Fall  2004  and 
Spring  2005)  to  complete  Phase  1  and  Phase  2  testing  of  all  600  subjects.  Accordingly,  we  asked 
for  and  received  a  one-year’s  extension  of  our  contract  with  Minerals  Management  Service  and 
ARI. 

Contributions  to  Basic  Research 

-  If  we  are  able  to  show  that  in  the  domain  of  numerical  reasoning  two  abilities,  speed  of 
numerical  reasoning  and  level  of  numerical  reasoning,  govern  success  during  time-constrained 
performance,  then:  (a)  we  will  be  the  first  researchers  to  have  validly  measured  a  person’s  speed 
of  performance  on  a  set  of  cognitive  tasks  having  non-trivial  difficulty;  (b)  we  will  have 
demonstrated  that,  within  a  given  task  domain,  the  conditions  of  task  performance  can  strongly 
influence  what  cognitive  abilities  are  measured;  (c)  we  will  have  found  a  method  to  more 
completely  and  accurately  summarize  a  person’s  proficiency  status  within  the  domain  of 
numerical  reasoning,  and  if  so,  we  expect  that  our  approach  would  likely  generalize  to  other 
cognitive  task  domains;  and  (d)  we  will  have  presented  strong  evidence  that  the  structure  of 
human  cognitive  abilities  is  much  more  complex  than  is  portrayed  by  the  most  widely  accepted 
factor-analytic  models  (Carroll,  1993;  Gustafsson,  1984);  those  models  are  based  on  time-limit 
test  scores,  scores  which  confound  level  ability  with  any  speed  ability  that  may  exist  for  a  given 
cognitive  domain. 

Potential  Army/Military  Applications 

Quickness  in  problem  solving  under  emergency  or  combat  conditions  is  a  highly 
desirable  characteristic  of  any  person  serving  in  a  command  or  key  technical  role  in  the  Army. 
The  desirability  of  this  characteristic  is  proportional  to  the  pace  at  which  technology  increases 
the  volume  and  speed  of  communications.  However,  none  of  the  four  subtests  that  comprise  the 
Armed  Forces  Qualification  Test  (AFQT)  provides  a  measure  of  performance  speed  in  any  of  the 
domains  the  AFQT  assesses.  Rather,  all  four  parts  are  intended  to  be  tests  of  power  (Russell, 
Peterson,  Rosse,  Hatten,  McHenry,  &  Houston,  2001,  p.  77).  Having  recruits’  speed  scores  in 
addition  to  their  level  scores  could  lead  to  important  improvements  in  matching  recruits  to  job 
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specializations,  particularly  in  cases  where  those  persons  would  need  to  be  able  to  perform 
complex  cognitive  tasks  not  just  successfully,  but  rapidly  as  well. 

Future  Plans 

This  project  was  extended  for  an  additional  year  because  of  needed  adjustments  to  the  testing 
procedures,  required  revisions  to  the  computer  software  used  to  record  orally-delivered  item 
responses,  and  competition  for  time  and  space  in  the  computerized-testing  laboratory  that 
resulted  in  lower  subject  participation  counts  than  necessary. 

Accordingly,  our  future  plans  include: 

•  Completing  recruitment  of  subjects  until  the  desired  sample  of  600  subjects  is  obtained. 

•  Completing  Phase  1  testing  of  the  balance  of  our  sample. 

•  Finishing  the  assignment  of  item-level  time  limits  so  that  Phase  2  testing  can  proceed.  For 
each  item,  it  is  necessary  to  compute  the  item’s  Phase  1  correct  response  time  5th  percentile, 
35th  percentile,  65th  percentile,  and  95th  percentile.  Items  from  each  form  will  then  be  sorted 
into  one  or  another  of  4  groups,  such  that  items  in  a  given  group  will  possess  similar  correct 
response  time  percentile  profiles.  Each  item  in  each  group  will  then  be  assigned  four  possible 
time  limits,  each  time-limit  being  equal  to  the  group’s  median  correct  response  time  5th, 
35th,  65th,  and  95th  percentile. 

•  Administering  Phase  2  testing  to  all  subjects  who  have  finished  Phase  1  testing.  Once  the 
items  have  been  assigned  to  the  four  item  groups,  subjects  will  be  randomly  assigned  to  one 
of  four  subject  subgroups.  In  Phase  2  testing,  each  subject  subgroup  will  answer  four  blocks 
of  items.  All  of  the  items  in  each  block  will  have  the  same  item-level  time  limit  (5th,  35th, 
65th,  or  95th  percentile). 

•  Using  the  Phase  1  data  to  estimate  each  subject’s  level  of  numerical  reasoning,  and  then 
using  those  level  scores  in  combination  with  the  Phase  2  data  to  estimate  each  subject’s 
speed  of  numerical  reasoning.  Finally,  we  will  compare  the  fits  of  our  competing 
measurement  models  (level-only  and  speed-level)  to  each  main  subject  group’s  Phase  2  data 
to  determine  whether  the  level -only  hypothesis  or  the  speed-level  hypothesis  is  better 
supported. 


65 


Modeling  the  Strategic  Role 
of  Basic  Emotions  with  Cellular  Automata 

Contract#:  W74Y8H-05-P-0005  Contract  Dates:  1/1/05-12/31/05 

Institution:  State  University  of  New  York,  Oswego  PI:  Craig  DeLancey 

Liaison:  Bob  Witmer,  SSRU 


Problem(s)/Research  Question(s)  -  Under  what  conditions  are  emotional  behaviors 
(particularly  retribution  in  anger  and  flight  in  fear),  which  in  the  short  term  may  appear 
irrational,  rational  or  beneficial  for  an  individual  or  their  group? 

Technical  Barrier(s)  -  Ethical  and  technical  difficulties,  along  with  prohibitive  costs,  create 
significant  impediments  to  human  subject  research  that  induces  and  observes  anger,  fear,  and 
other  emotions  in  controlled  situations. 

Significance/Impact  for  Basic  Research  -  The  research  will  provide  a  model  that  will  help  us 
understand  how  complex  emotional  behaviors  can  be  beneficial  or  detrimental  in  both  individual 
and  social  contexts.  This  can  provide  predictions  for  practical,  more  focused  human  subject 
research. 

Potential  Transitions  -  Knowledge  to  be  gained  in  this  basic  research  may  be  usefully 
employed  in  studies  including  but  not  limited  to: 

•  ATO  III.HS.2004.01  -  Accelerating  Leader  Development  (WP  269  &  285) 

•  LEADERSHIP:  Developing  Leaders  in  a  Changing  Army  (WP  1 03). 

•  Assessing  and  Developing  Leaders  for  Special  Operations  Units  (WP  218). 

Overview 

The  goal  of  this  research  is  to  develop  a  model  of  the  minimal  conditions  for  certain  kinds  of 
emotional  behaviors  -  particularly  flight  in  fear  and  retribution  in  anger  -  to  occur  and  to  be 
beneficial  for  an  individual  or  for  a  group.  The  model  will  also  produce  predictions  that  can  in  turn 
be  tested  in  future  empirical  research,  including  empirical  research  with  humans. 

Emotions  often  motivate  behaviors  that  are  essential  for  our  survival,  but  sometimes  they  can 
lead  to  behaviors  that  appear  irrational.  Some  of  these  seemingly  irrational  behaviors  may  in  fact  be 
irrational,  but  others  may  be  behaviors  that  on  balance  benefit  us  if  we  understand  them  in  the  right 
context.  We  have  as  of  yet  little  ability  to  distinguish  these  cases,  or  when  they  are  likely  to  happen. 
Important  tools  for  predicting  and  evaluating  the  decision  making  of  individuals  have  been 
developed  in  economics  and  related  fields  of  inquiry.  In  general,  these  tools  are  based  on  the 
assumption  that  individuals  are  rational  agents  who  evaluate  the  expected  outcome  of  the  choices 
before  them,  and  then  make  choices  that  lead  to  their  most  immediate  benefit.  However,  actual 
empirical  research  into  human  decision  making  reveals  some  surprising  behaviors  inconsistent  with 
a  simple  interpretation  of  these  kinds  of  theories.  An  early  and  surprising  result  of  experimental 
economics  was  the  discovery  of  an  unpredicted  and  seemingly  irrational  behavior  in  ultimatum 
games  that  appears  to  be  caused  by  anger  or  another  retributive  affect  (Roth  1995).  An  ultimatum 
game  is  a  situation  in  which  one  individual  is  given  some  utility  (typically  money)  and  that  agent 
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must  decide  how  to  share  this  utility  with  another  agent.  The  receiving  agent  has  no  say  in  the 
distribution,  but  can  decide  to  “veto”  the  offer,  resulting  in  neither  individual  getting  anything. 
Traditional  economics,  based  upon  a  simple  decision-theoretic  analysis,  predicts  that  the  receiver 
will  never  exercise  veto,  but  will  rather  accept  any  offer,  since  something  is  better  than  nothing;  and 
predicts  that  the  distributing  agent,  knowing  this,  will  always  distribute  as  little  as  allowed  in  the 
game,  and  retain  the  rest.  However,  in  actual  experimental  tests  with  human  subjects,  the  veto  is 
frequently  used  in  response  to  minimal  distributions.  Thus,  it  appears  that  people  often  prefer  less 
utility  for  themselves  in  order  to  punish  others  whom  they  perceive  as  making  unfair  distributions. 
One  interpretation  of  this  behavior  is  that  individuals  undertake  retributive  behaviors  because  they 
are  angry.  It  is  a  likely  hypothesis  that  such  behavior  may  be  rational  in  the  long-term  if  it  tends  to 
enforce  more  equitable  distribution  for  either  that  individual,  or  for  their  social  group.  Similar  kinds 
of  seemingly  irrational,  but  rational  behaviors  may  arise  from  fear,  where,  for  example,  a  fast 
processing  of  threats  can  benefit  agents  confronted  with  a  great  deal  of  information  and  who  need  to 
quickly  avoid  threats,  but  this  ability  can  lead  to  actions  detrimental  in  the  short  term. 

Both  retributive  and  flight  behaviors  also  appear  to  have  an  important  social  element.  Agents 
may  be  more  likely  to  be  afraid  in  a  peer  group  where  others  are  afraid  (Hatfield,  Cacioppo,  Rapson 
1993),  and  retributive  behavior  appears  to  be  more  common  when  individuals  in  groups  perceive 
that  the  group  (not  the  individual)  is  responsible  (Milgram  1974).  Both  kinds  of  cases  are  consistent 
with  a  hypothesis  that  some  emotions,  such  as  fear  and  anger,  can  be  both  excited  and  inhibited  by 
the  observable  emotions  of  others  in  one’s  social  group.  Recent  analysis  has  also  raised  the 
possibility  that  individuals  determine  whether  to  express  an  emotion  or  suppress  it  based  on 
evaluation  of  the  expected  social  outcome  (Griffiths  2003).  Such  factors  ensure  that  group  behavior 
can  be  significantly  different  from  individual  behavior. 

The  general  notion  that  emotions  have  a  special  role  to  play  in  strategic  action  has  received 
growing  confirmation  in  a  range  of  other  sciences.  Neuropsychological  studies  of  ultimatum  game 
performance  have  confirmed  the  role  of  brain  substrates  of  emotion  in  ultimatum  games  and  other 
kinds  of  games  (Sanfey  et  al  2003,  Montague  and  Bems  2002).  Emotional  deficits  arising  from 
brain  damage  have  been  identified  that  result  in  poor  gambling  and  strategic  choice  behavior 
(Anderson  et  al  1999;  Bechara  et  al  1994).  The  neural  substrates  of  fear  have  become  well 
understood  (Panksepp  1998,  LeDoux  1984),  and  the  effect  of  fear  on  attention  and  other  capabilities 
required  for  decision  making  are  increasingly  being  investigated  (Bishop  et  al  2004). 

However,  progress  in  psychology  and  neural  sciences  fails  to  address  two  important  features 
of  the  role  of  emotions  in  strategic  action.  First,  existing  models  typically  have  no  way  to  account 
for  a  benefit  of  emotional  behaviors  that  stretches  across  games,  or  that  influences  the  social  context 
that  in  turn  determines  the  structure  of  future  games.  For  example,  one  utility  of  retributive  behavior 
may  be  to  enforce  social  conventions  (such  as  fairness  in  various  distributions)  that  benefit  on 
average  the  individual  in  question.  Exploring  such  a  possibility  requires  some  modeling  of  social 
interactions  over  both  a  range  of  games,  but  also  in  a  context  where  behavior  shapes  future 
interactions.  Second,  there  is  insufficient  attention  to  the  complex  social  interactions  that  such 
emotions  may  require.  We  may  need  to  explain  the  role  and  utility  of  a  basic  emotion  in  an  agent’s 
action  not  only  in  terms  of  how  it  may  have  utility  to  the  agent  either  immediately  or  by  helping  to 
maintain  certain  social  conventions  from  which  the  agent  benefits,  but  also  in  terms  of  how  the 
display  of  the  emotion  affects  other  agents. 
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Experimental  conditions  and  existing  models  are  not  yet  able  to  address  such  possibilities. 
Furthermore,  we  cannot  expect  psychology  or  neural  sciences  to  explore  these  problems  in  the  near 
term  with  animal-subject  experiments  or  human-subject  experiments,  because  the  requirements  of 
appropriate  experiments  are  prohibitive.  It  is  very  difficult  to  induce  fear,  anger,  and  other  emotions 
reliably  and  with  controlled  and  significant  intensities  in  subjects.  It  is  even  more  difficult  to  induce 
such  emotions  in  only  selected  subjects  in  groups.  Finally,  it  is  cost-prohibitive  to  generate 
sufficient  test  cases  to  control  for  the  large  error  one  can  expect  in  subject  interactions  because  of 
their  inherit  complexity.  For  these  and  other  reasons,  the  strategic  role  of  basic  emotions  will  remain 
intractable  to  human  and  animal  experiments  for  some  time  to  come. 

Research  Approach 

The  project  will  use  a  technique  called  genetic  algorithms  to  develop  and  test  possible 
strategies  that  simple  agents  may  use  in  a  range  of  games  or  simple  strategic  situations.  This 
technique  allows  for  the  evolution  of  solutions  to  very  complex  problems.  The  problems  to  be 
answered  in  this  case  are  concerned  with  when  it  is  beneficial  to  adopt  flight  or  retributive 
behavior.  A  series  of  hypotheses  about  the  conditions  for  the  evolution  of  such  behavior  will  be 
tested  under  conditions  of  varied  games  (strategic  situations)  and  varied  capabilities  for  the 
agents. 

Accomplishments 

NA  -  New  Start. 

Contributions  to  Basic  Science 

The  project  will  provide  a  new  approach  to  modeling  the  strategic  role  of  emotions  that 
augments  and  extends  standard  game-theoretic  or  decision-theoretic  approaches.  The 
interpretation  of  the  final  model  and  results  can  result  in  a  new  set  of  ways  to  conceive  of  the 
role  of  several  important  emotional  behaviors. 

Potential  Army/Military  Applications 

Utilizing  and  controlling  fear  and  anger  are  essential  parts  of  the  soldier’s  performance, 
especially  in  combat  situations.  Understanding  fear  and  anger  can  also  have  a  significant  role  to 
play  in  understanding  and  predicting  the  actions  of  opponents.  The  working  hypothesis  of  the 
research  is  that  some  emotional  behaviors  can  be  better  understood  with  a  model  that  describes  the 
individual  and  social  conditions  in  which  they  can  be  beneficial.  This  can  also  explain  conditions 
under  which  some  emotional  behaviors  can  be  detrimental,  and  provide  predictions  of  what 
generates  such  emotional  behaviors. 

The  model  will  provide  empirical  and  conceptual  benefits.  First,  predictions  from  the  model 
could  be  tested  with  empirical  human  subject  research.  Ideally,  empirical  confirmation  of  the 
simulation  model  would  then  allow  for  the  determination  of  conditions  that  would  encourage 
beneficial  emotional  behaviors  and  discourage  detrimental  ones;  or,  if  desired,  vice  versa.  For 
example,  confirmation  of  the  existence  of  a  reliable  strategic  role  for  contagion  of  emotions  (the 
positive  influence  of  one  individual’s  emotions  on  the  predisposition  of  others  in  the  peer  group  to 
have  that  same  emotion)  could  provide  an  improved  understanding  of  when  it  would  be  useful  to 
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encourage  the  display  of  emotions.  Second,  the  model  could  also  provide  a  set  of  conceptual  tools  to 
enable  individuals  to  understand  emotional  behaviors.  For  example,  provided  with  new  ways  to 
understand  how  fear  and  anger  benefit  and  harm  us,  and  how  they  act  in  social  situations,  officers 
would  be  empowered  to  better  motivate  or  predict  the  actions  of  those  under  their  command. 
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A  Valid,  Culture-Fair  Test  of  Intelligence 

Contract  #:  New  Start.  Not  yet  assigned.  Contract  DATES:  To  be  determined. 

Institution:  Case  Western  Reserve  University  PI:  Joseph  F.  Fagan 

Liaison:  Peter  Legree,  RACO 

Problem(s)/Research  Question(s)  -  Can  a  racially  unbiased  test  of  the  ability  to  process 
information  predict  how  well  young  adults  will  succeed  in  college  classes? 

Technical  Barrier(s)  -  Current  theories  of  intelligence  are  based  on  an  assumption  that  all  those 
taking  IQ  tests  have  had  equal  opportunity  for  exposure  to  the  information  being  tested.  Thus, 
past  efforts  to  develop  an  intelligence  test  that  is  culture-fair  have  not  been  successful. 

Significance/Impact  for  Basic  Research  -  This  research  will  test  the  theory  that  intelligence  is 
information  processing  ability  (Fagan,  1992,  2000).  Specifically,  how  much  you  know  (your  IQ) 
depends  not  only  on  how  well  you  process  information  (your  intelligence)  but  also  on  the 
information  given  to  you  by  your  culture  to  process.  Current  research  on  a  theory  of  intelligence 
as  information  processing  finds  racial  differences  in  IQ  to  be  due  to  cultural  factors.  A  test  of 
information  processing  promises  to  be  the  first  valid,  culture-fair  test  of  intelligence. 

Potential  Transitions  -The  results  of  this  research  will  aid  in  gaining  the  objectives  of: 

•  Future  Force  Warrior  Training  (WP  2 1 5) 

•  Training  Small  Unit  Leaders  and  Teams  (ATO:  IV.HS.  2003.06) 

•  SELECT21:  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force 
Soldier  (WP  257)  and  ATO:  IV.HS.  2002.01. 

•  LEADERSHIP:  Developing  Leaders  in  a  Changing  Army  (WP  103) 

•  FUTURE-TRAIN:  Techniques  and  Tools  for  C3ISR  Training  of  Future  Brigade  Combat 
Team  Commanders  and  Staffs  (WP  211) 

•  Assessing  and  Developing  Leaders  for  Special  Operations  Units  (WP  218) 

•  Accelerating  Leader  Development  (WP  269);  Accelerating  Leader  Development  (WP  285). 

Overview 

The  purpose  of  this  research  is  to  test  the  predictive  validity  of  a  racially  unbiased  test  of 
intelligence.  The  test  is  based  on  the  ability  to  process  new  information.  Preliminary  work 
indicates  that  this  new,  culture-fair  test  of  intelligence  is  as  predictive  of  academic  achievement 
as  are  standard  academic  aptitude  tests.  Differences  in  the  intelligence  quotient  (IQ)  between 
Blacks  and  Whites  on  the  order  of  about  15  points  are  well  documented.  Jensen  (1998)  considers 
such  racial  differences  in  IQ  to  be  due  to  differences  in  basic  intellectual  ability.  In  making  this 
argument,  Jensen  assumes  that  the  opportunity  for  exposure  to  the  information  being  tested  on 
conventional  IQ  tests  has  been  the  same  for  Blacks  and  Whites.  In  contrast  to  Jensen,  Fagan 
assumes  (Fagan,  1992;  Fagan,  2000;  Fagan  &  Haiken-Vasen,  1997)  that  the  IQ  score  is  a 
measure  of  knowledge.  How  much  a  person  knows  (their  IQ)  depends  on  the  person’s 
information  processing  ability  and  on  the  information  the  person  has  been  given  to  process.  In 
this  view,  if  group  differences  in  IQ  are  not  accompanied  by  group  differences  in  information 
processing,  then  the  search  for  the  causes  of  the  IQ  differences  should  be  directed  toward 
differences  in  access  to  information. 
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Fagan  &  Holland  (2002)  investigated  the  contributions  that  intellectual  ability  and  access 
to  information  make  to  racial  differences  in  IQ.  Blacks  and  Whites  (community  college 
students)  were  compared  on  their  knowledge  of  the  meanings  of  words,  a  task  that  typically 
results  in  racial  group  differences  in  IQ.  The  Black  and  White  participants  in  the  Fagan  & 
Holland  (2002)  studies  were  highly  representative  of  the  U.S.  population  with  regard  to  age  and 
educational  level  (U.  S.  Census  Bureau,  1999).  Fagan  &  Holland  (2002)  insured  that  Blacks  and 
Whites  were  given  equal  opportunity  to  learn  the  meanings  of  novel  words  and  conducted  tests 
to  determine  how  much  knowledge  had  been  acquired.  For  example,  participants  might  be 
exposed  to  a  sentence  such  as  “Tubby  had  a  big  fat  venter”  and  asked  to  indicate  whether  a 
“venter”  was  a  body  part  or  a  mental  state.  On  a  later  test  of  vocabulary  knowledge,  the  word 
"venter"  would  be  presented  with:  “a.  height,  b.  candle,  c.  badge,  d.  belly,  and  e.  opening”  as 
choices.  Fagan  &  Holland  (2002)  controlled  for  the  possibility  that  the  particular  people  chosen 
to  represent  each  racial-ethnic  group  might,  by  chance,  simply  have  been  equal  in  vocabulary 
knowledge  to  begin  with.  Thus,  the  tests  of  a  person's  knowledge  of  the  newly  learned  words 
were  intermixed  with  similar  multiple  choice  tests  for  knowledge  of  the  meanings  of  different 
words,  words  for  which  no  special  training  in  the  experimental  situation  had  been  provided.  The 
choices  for  a  word  more  frequent  in  English  such  as  "situate"  (to  place),  for  example,  were: 
“wear,  add,  take,  study,  and  place”.  Whites  were,  as  expected,  superior  to  the  Blacks  in  untrained 
vocabulary  knowledge.  However,  when  equal  opportunity  for  exposure  to  the  meanings  of  words 
was  experimentally  assured,  Whites  and  Blacks  were  equal  in  vocabulary  knowledge.  Fagan  & 
Holland  (2002)  concluded  that  Blacks  and  Whites  do  not  differ  in  the  ability  to  process  new 
information  and  that  the  search  for  racial  differences  in  knowledge  (IQ)  should  be  aimed  at 
differences  in  the  information  to  which  people  from  different  racial  groups  have  been  exposed. 

In  a  second  series  of  studies  (Fagan  &  Holland  2003),  Blacks  and  Whites  were  again 
compared  for  their  knowledge  of  items  commonly  employed  in  tests  of  IQ,  i.e.,  the  meanings  of 
sayings,  of  similarities,  and  of  analogies.  The  participants  were  later  tested  as  to  the  extent  of 
their  newly  learned  knowledge.  For  example,  participants  learned  that  the  saying  “fry  asparagus” 
means  “get  lost,  take  a  hike”.  Later  they  were  asked  if  “fry  asparagus  meant  to  plant  seed,  to 
make  supper,  to  come  again,  or  to  go  away”.  They  also  learned  that  “An  ancient  animal  has  been 
found  who  saw  with  one  eston  and  listened  with  one  ewot.”  Later  they  were  asked  how  “eston 
and  ewot”  were  most  similar:  “a.  body  parts,  b.  start  with  E,  c.  relatives,  d.  senses”.  A  second 
group  of  students  were  presented  with  sayings,  similarities  and  analogies  based  on  specific 
previously  acquired  knowledge.  A  saying  based  on  such  specific  information  might  be: 
BENEDICT  ARNOLD  a.  President  b.  Patriot  c.  Monk  d.  Traitor.  If  Blacks  and  Whites  are 
equally  able  to  process  information,  they  should  be  equally  able  to  answer  questions  based  on 
newly  learned  information.  If  equal  access  to  specific  information  is  not  culturally  available, 
Blacks  and  Whites  should  differ  in  answering  questions  based  on  specific  information. 
Comprehension  based  on  newly  learned  information  (whether  for  sayings,  similarities,  or 
analogies)  did  not  vary  by  race.  At  the  same  time,  knowledge  based  on  specific  information  (no 
matter  the  material  tested)  was  higher  for  the  Whites  than  for  the  Blacks.  Thus,  when 
information  was  equally  available,  Whites  and  Blacks  were  equal  in  knowledge. 

The  data  of  Fagan  &  Holland  (2002,  2003)  support  the  view  that  cultural  differences  in 
the  provision  of  information  may  account  for  racial  differences  in  IQ.  Specifically,  the  results 
indicate  that  IQ  differences  between  Blacks  and  Whites  have  to  do  with  experience.  The 
proposed  research  focuses  on  the  implications  that  the  results  of  the  Fagan  and  Holland  studies 
have  for  the  development  of  a  valid,  culture-fair  test  of  intelligence.  In  fact,  a  preliminary  study 
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indicates  that  this  new,  culture-fair  test  of  intelligence  is  as  valid  in  predicting  scholastic 
performance  as  is  a  standard  academic  aptitude  test. 

Research  Approach 

Brief  tests  of  knowledge  based  on  word  meanings,  similarities,  sayings,  and  analogies 
which  have  been  proven  to  be  culture-fair  will  be  administered  to  college  students  18  years  or 
older.  Students,  in  groups,  will  be  given  multiple  choice  tests  of  their  ability  to  acquire  new 
information  concerning  the  meanings  of  previously  unknown  words,  sayings,  similarities,  and 
analogies.  They  will  also  be  tested  for  their  knowledge  of  vocabulary  in  general,  opposites,  and 
analogies.  An  entire  session  (training  and  testing)  will  last  about  30  minutes.  With  the  student’s 
written  consent,  Scholastic  Assessment  Test  scores  or  ACT  scores,  GPA’s  and  numerical  grades 
in  the  Psychology  course  will  also  be  obtained.  Class  instructors  (with  the  student’s  written 
consent)  will  be  asked  to  provide  the  student’s  numerical  scores  on  the  tests  in  the  course  but  not 
the  final  letter  grade.  Associations  between  performance  on  the  culture-fair  tests,  academic 
aptitude  (Scholastic  Assessment  Test  scores),  and  specific  achievement  (cumulative  GPA’s  and 
objective  test  scores  in  college  courses)  will  be  analyzed.  The  proposal  presented  here  is 
currently  approved,  through  expedited  review,  by  the  CWRU  Institutional  Review  Board  for 
Protocol  #20031104  titled  “New  Knowledge  2”  (J.  F.  Fagan,  PI)  which  can  be  renewed  on  a 
yearly  basis  for  a  period  of  four  years.  (See  attached  current  approval). 

Accomplishments  -  N/A-New  Start 

Contributions  to  Basic  Science 

Jensen  (1998)  believes  that  the  IQ  score  reflects  what  he  calls  the  general  factor  (or  g ) 
underlying  intelligence.  Differences  in  IQ  scores  are  due  to  differences  in  g.  According  to 
Jensen,  genetic  factors  play  a  primary  role  in  determining  g.  Thus,  Jensen  (2000)  believes  that  it 
may  not  be  possible  to  come  up  with  tasks  that  show  no  differences  in  test  performance  between 
Blacks  and  Whites  and  yet  still  predict  academic  performance.  Fagan  (2000)  does  not  agree  with 
Jensen,  pointing  out  that  the  failure  to  develop  tests  of  intelligence  that  can  be  fairly  applied 
across  racial  groups  stems  from  a  theoretical  bias  to  equate  the  IQ  score  with  intelligence  rather 
than  with  knowledge.  If  intelligence  is  defined  as  information  processing  and  the  IQ  score  as 
knowledge,  the  possibility  of  culture-fair  tests  of  intelligence  based  on  estimates  of  information 
processing  arises.  Fagan  is  not  alone  in  such  a  conjecture.  Williams  (2000,  p.l  7)  notes  that 
“Fagan’s  ideas”  (Fagan  2000)  of  measuring  thinking  or  information  processing  rather  than 
accumulated  knowledge  are  relevant  to  the  debate  on  intelligence  testing  and  affirmative  action 
because  ...  a  true  measure  of  processing  efficiency  (if  it  could  be  devised)  would  be  fair  to 
members  of  all  racial  and  ethnic  groups”.  The  chief  practical  implication  of  the  Fagan  & 
Holland  studies  noted  above  is  that  it  may  be  possible  to  develop  culture-fair  tests  of  intelligence 
that  will  allow  basic  intellectual  abilities  to  be  recognized.  Specifically,  the  inequality  of 
educational  achievement  among  races  in  our  country  has  highlighted  a  need  for  culture-fair  tests 
of  intelligence.  For  example,  Blacks  do  not  do  as  well  as  Whites  on  IQ  tests  and  other  tests  of 
knowledge  such  as  the  SAT.  Basing  admission  to  higher  education  on  such  test  scores  means 
that  only  a  small  percentage  of  Blacks  are  eligible  for  admission  to  colleges  and  universities.  A 
culture-fair  test  of  intelligence  would  allow  basic  abilities  to  be  measured  and  would  allow  those 
with  appropriate  intellectual  skills  to  pursue  further  schooling. 
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Potential  Army/Military  Applications 

Practically,  culture-fair  tests  of  information  processing  that  are  valid  predictors  of 
achievement  have  important  military  application.  Such  tests  can  aid  in  providing  a  valid, 
culturally  unbiased  means  to  select  candidates  for  advanced  education,  employment,  or  “training 
in  complex  situations,  including  factors  [aptitudes]  that  enhance  transfer  of  training  of  the  sort  of 
complex  tasks  performed  by  Army  personnel”  (ARI  BAA,  p.2).  Specifically,  the  Army  can  use 
such  a  measure  as  an  adjunct  to  their  standard  measures,  initially,  to  evaluate  volunteers.  Such 
tests  can  also  be  employed  in  further  selection  and  training  stages.  In  the  field,  ready 
information  processing  skills  are  also  necessary  to  make  quick  and  correct  decisions  based  on 
rapidly  incoming  information.  Basic  information  processing  ability  is  also  a  key  aptitude  in  the 
selection  of  future  army  leaders.  Finally,  the  use  of  tests  known  to  be  culture-fair  may  provide 
an  incentive  to  recruitment  and  re-enlistment. 

Future  Plans  -  N/A  -  New  Start 


73 


Testing  a  Theory  of  the  Determinants  of  Individual  Job  Performance  for 
United  States  Army  Junior  Commissioned  Officers 

Contract  #:  W74V8H-05-K-0005  Contract  Dates:  1/10/05-8/31/06 

Institution:  Personnel  Decisions  Research  Institutes,  Inc.  PI:  Jeff  W.  Johnson 

Liaison:  Robert  Kilcullen,  SARU 

Problem(s)/Research  Question(s)  -  How  do  individual  difference  variables  work  to  influence 
performance  on  specific  performance  dimensions  relevant  to  junior  commissioned  officers  in  the 
U.  S.  Army? 

Technical  Barrier(s)  -  There  is  no  well-established  theory  of  the  process  by  which  individual- 
difference  variables  predict  citizenship  performance  or  adaptive  performance,  in  either  a  military 
or  a  civilian  context. 

Significance/Impact  for  Basic  Research  -  Support  for  this  theory  of  performance  would 
contribute  to  a  better  understanding  of  the  relationships  between  predictors,  mediators,  and  job 
performance  criteria. 

Potential  Transitions  -  ARI  has  created  various  temperament  scales  thought  to  be  related  to 
adaptive  performance  and  empirically  linked  these  measures  to  the  mission  performance  of 
Special  Forces  (SF)  Soldiers.  ARI  is  now  leveraging  this  measure  to  create  a  self-development 
feedback  packet  for  SF  Officers.  The  measures  created  in  this  basic  research  project  are  designed 
to  predict  performance  at  a  more  proximal  level  than  the  current  temperament  scales,  so  they 
could  be  used  to  revise  or  improve  this  self-development  package  or  other  Soldier  self¬ 
development  packages  that  are  used  outside  of  SF. 

Overview 

Organizational  citizenship  performance  and  adaptive  performance  are  increasingly 
important  components  of  the  Army  junior  commissioned  officer  job  performance  construct 
domain.  Despite  the  importance  of  these  constructs,  there  has  been  very  little  research  studying 
them  in  a  military  setting.  There  is  also  no  well-established  theory  of  the  process  by  which 
individual-difference  variables  predict  citizenship  performance  or  adaptive  performance,  in 
either  a  military  or  a  civilian  context.  In  this  research,  we  will  test  a  theory  of  the  process  through 
which  individual  difference  variables  work  to  influence  performance  on  specific  citizenship  and 
adaptive  performance  dimensions  relevant  to  junior  commissioned  officers  in  the  U.  S.  Army. 
Johnson  (2003)  proposed  a  theory  of  performance  determinants  that  goes  beyond  earlier  theories 
proposed  by  Campbell,  McCloy,  Oppler,  and  Sager  (1993)  and  Motowidlo,  Borman,  and  Schmit 
(1997),  primarily  by  specifying  an  expanded  conceptualization  of  motivation.  Johnson  broke  the 
motivation  component  into  specific  motives  (e.g.,  organizational  commitment,  job  satisfaction), 
proactive  cognitive  processes  (e.g.,  self-efficacy,  goal  commitment),  and  on-line  cognitive 
processes  (e.g.,  self-regulation).  These  variables,  along  with  knowledge,  skill,  and  habits, 
mediate  the  relationship  between  indirect  performance  determinants  (e.g.,  cognitive  ability, 
personality)  and  any  kind  of  individual  job  performance  construct. 

Support  for  this  theory  would  provide  researchers  and  practitioners  with  a  tool  that  can  be 
used  to  identify  the  constructs  through  which  individual  difference  variables  work  to  influence 
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performance  on  specific  dimensions.  This  is  especially  important  for  identifying  predictors  of 
citizenship  and  adaptive  performance.  Such  a  tool  would  contribute  to  a  better  understanding  of 
the  relationships  between  predictors,  mediators,  and  job  performance  criteria,  and  could  be  used 
to  choose  appropriate  predictors  for  whatever  criterion  construct  is  of  interest  for  a  particular 
Army  job.  A  rigorously  developed  and  tested  theory  of  the  influence  of  individual  differences  on 
job  performance  would  not  only  facilitate  selection,  classification,  and  training  of  junior 
commissioned  Army  officers,  but  would  also  add  significantly  to  the  evolving  understanding  of 
job  performance. 

Research  Approach 

In  order  to  test  this  theory,  we  will  assemble  and  develop  a  battery  of  instruments  that  are 
construct-valid  measures  of  each  component  of  the  theory.  All  measures  created  for  this  project 
will  be  based  on  input  from  ROTC  cadets.  To  measure  task,  citizenship,  and  adaptive 
performance,  we  will  create  a  multisource  performance  rating  instrument  measuring  performance 
on  five  different  dimensions.  We  will  also  create  a  past  behavior  record  to  measure  skill  and  a 
situational  judgment  test  to  measure  knowledge,  both  of  which  will  be  relevant  to  five 
performance  dimensions.  We  will  create  (a)  a  unique  self-report  measure  of  work  habits;  (b)  a 
motives  scale  assessing  attitudes,  values,  and  preferences  that  predict  performance;  and  (c)  a 
motivation  scale  measuring  self-efficacy,  expectancy,  goal  content,  and  goal  commitment.  We 
will  also  use  existing  instruments  to  measure  cognitive  ability,  personality,  experience,  and 
action  control. 

-  The  predictor  battery  will  be  administered  to  200  ROTC  cadets  near  the  beginning  of  the 
semester.  Criterion  administration  will  occur  near  the  end  of  the  semester  so  we  can  assess  the 
extent  to  which  cadets  performed  the  behaviors  they  intended  to  perform.  Each  cadet  will  be 
rated  by  approximately  four  peers  and/or  ROTC  cadre/staff.  These  data  will  be  analyzed  using 
structural  equation  modeling  to  test  the  Johnson  (2003)  theory. 

Accomplishments 

NA  -  New  Start. 

Contributions  to  Basic  Science 

This  project  will  advance  our  understanding  of  job  performance  prediction  by  providing 
evidence  in  support  of  a  theory  of  the  process  by  which  individual-difference  variables  predict 
specific  dimensions  of  performance.  Many  theories  of  the  relationship  between  individual 
differences  and  job  performance  have  been  proposed,  but  there  is  very  little  empirical  support  for 
any  of  them.  This  is  especially  true  for  less-studied  aspects  of  performance  such  as  citizenship 
performance  and  adaptive  performance,  which  are  the  primary  focus  of  this  research.  An 
established  model  of  the  process  by  which  individual  differences  in  predictor  variables  lead  to 
individual  differences  in  specific  dimensions  of  performance  can  be  used  to  guide  research 
linking  specific  predictors  to  specific  performance  dimensions  by  helping  to  identify 
theoretically  relevant  predictors  for  different  criteria.  In  the  proposed  model,  the  construct  of 
motivation  is  expanded  to  highlight  how  different  predictors  influence  different  components. 
Self-regulation  is  the  primary  component  that  previous  models  were  missing.  This  construct  is 
very  important  because  it  (a)  is  strongly  related  to  personality;  (b)  helps  explain  how  people  with 
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similar  knowledge,  ability,  goals,  and  desire  to  perform  differ  in  their  level  of  performance;  and 
(c)  helps  explain  how  people  overcome  conflicting  habits  to  perform  in  accordance  with  their 
goals. 


This  project  will  also  yield  several  potentially  useful  products.  We  will  create  a 
multisource  performance  rating  instrument  to  measure  task,  citizenship,  and  adaptive 
performance.  These  performance  components  are  important  aspects  of  the  Army  junior 
commissioned  officer  performance  construct  domain.  We  will  create  a  situational  judgment  test 
(SJT)  to  assess  knowledge  relevant  to  the  performance  dimensions  chosen  for  study.  The  SJT 
will  present  the  individual  with  a  situation  and  a  set  of  response  options  varying  in  their 
effectiveness.  We  will  develop  a  past  behavior  record  to  measure  skill  relevant  to  the  targeted 
performance  dimensions.  Using  a  highly  structured  format  to  minimize  the  possible  confounding 
effect  of  writing  ability,  cadets  will  describe  a  recent  situation  they  experienced  that  elicited 
behavior  relevant  to  the  performance  dimension  of  interest  and  the  behavior  they  exhibited. 
Trained  raters  will  use  standardized  rating  scales  to  evaluate  the  level  of  skill  that  behavior 
represents.  By  allowing  respondents  to  describe  their  best  example  of  behavior  relevant  to  a 
performance  dimension,  we  can  assess  the  extent  to  which  the  person  is  able  to  exhibit  the 
behavior  irrespective  of  the  extent  to  which  they  typically  exhibit  the  behavior.  To  measure  the 
components  of  motivation  described  in  Johnson’s  (2003)  model,  we  will  create  (a)  a  unique  self- 
report  measure  of  work  habits;  (b)  a  motives  scale  assessing  attitudes,  values,  and  preferences 
that  predict  performance;  (c)  a  cognitive  processes  motivation  scale  measuring  self-efficacy, 
expectancy,  goal  content,  and  goal  commitment;  and  (d)  a  self-report  measure  of  self-regulation 
with  respect  to  targeted  behaviors. 

Potential  Army/Military  Applications 

Support  for  the  performance  model  will  provide  the  Army  with  a  tool  that  can  be  used  to 
identify  the  constructs  through  which  individual  difference  variables  work  to  influence 
performance  on  specific  dimensions.  This  is  especially  important  for  citizenship  and  adaptive 
performance,  which  are  important  components  of  Army  officer  jobs  but  have  received  little 
research  attention  in  a  military  or  civilian  context.  Such  a  tool  would  contribute  to  a  better 
understanding  of  the  relationships  between  predictors,  mediators,  and  job  performance  criteria. 
For  the  Army,  this  tool  would  probably  have  its  best  application  in  identifying  training  and/or 
development  needs.  Given  a  criterion  construct  on  which  an  individual’s  performance  is  in  need 
of  improvement,  this  model  can  help  to  identify  the  determinants  of  performance  on  that 
construct.  For  example,  an  individual  possessing  adequate  skill  and  knowledge  may  determine 
that  he  or  she  must  learn  new  self-regulatory  strategies  to  overcome  counterproductive  task- 
related  habits. 

This  project  will  yield  several  products  that  may  be  very  useful  to  the  Army.  First,  the 
various  motivation-related  instruments  we  will  develop/assemble  and  validate  against  criteria  of 
importance  to  the  Army  should  prove  to  be  quite  useful  as  self-development  tools.  Completing 
these  instruments  can  help  soldiers  and/or  officers  learn  about  their  own  attitudes,  values,  goal 
commitment,  self-efficacy,  self-regulatory  skills,  and  habits  relevant  to  five  critical  performance 
dimensions. 

Second,  the  Past  Behavior  Record  Form  may  prove  to  be  useful  for  training.  In  particular, 
the  evaluation  guide  that  would  be  used  to  score  this  instrument  will  provide  useful  information 
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regarding  where  various  skill-relevant  behaviors  fall  on  various  performance  continua. 
Moreover,  the  principles  to  be  induced  during  the  process  of  calibrating  the  performance¬ 
relevant  skills  to  different  points  on  these  anchored  rating  scales  will  make  useful  learning  points 
in  training  programs.  The  situational  judgment  test  will  also  be  a  useful  self-paced  training  tool. 
Individuals  are  presented  with  realistic  situations  and  choose  from  a  variety  of  alternative 
response  options.  Explanations  of  the  strengths  and  weaknesses  of  each  response  can  be 
developed  to  help  the  trainee  understand  the  level  of  effectiveness  of  the  chosen  option  and  the 
most  effective  option. 

Finally,  the  performance  rating  form  may  have  applied  value  for  performance  appraisal, 
training  needs  analysis,  or  criteria  for  future  validation  studies.  In  addition,  it  would  make  a 
useful  feedback  and  development  tool.  The  performance  rating  form  will  be  a  multi-source  rating 
instrument.  As  such,  it  will  not  only  provide  a  profile  of  strengths  and  weaknesses,  it  will 
provide  information  about  discrepancies  between  how  an  individual  is  seen  from  different 
perspectives  (i.e.,  subordinates,  peers,  superiors).  It  would  be  extremely  useful,  for  example,  for 
a  junior  officer  to  become  aware  of  differences  between  his  or  her  self-perceived  performance 
and  the  perceptions  of  his  or  her  performance  held  by  subordinates. 

Future  Plans 

During  fiscal  year  2005,  we  will  conduct  the  following  activities: 

•  Conduct  kick-off  meeting:  We  will  hold  a  kickoff  meeting  with  ARI  personnel  to 
discuss  the  work  plan  and  to  refine  or  redirect  our  work  activities. 

•  Obtain  participants  for  instrument  development  workshops:  We  will  contact  ROTC 
unit  commanding  officers  to  arrange  for  participants.  We  will  then  recruit  cadets  and 
schedule  them  for  the  workshops  in  which  they  express  interest. 

•  Define  performance  domain  and  create  criterion  measure:  We  will  identify  five 
dimensions  of  performance  representing  task,  citizenship,  and  adaptive  performance 
that  are  relevant  to  ROTC  cadets.  Then  we  will  develop  a  performance  rating  form  to 
measure  performance  on  these  dimensions. 

•  Identify  and  obtain  predictor  measures:  We  will  assemble  a  battery  of  predictor 
measures  that  includes  measures  of  general  cognitive  ability,  personality,  and 
experience. 

•  Conduct  situation  generation  workshops  for  developing  knowledge  measure:  We  will 
conduct  two  situation  generation  workshops  in  which  cadets  will  be  presented  with 
definitions  of  the  target  dimensions.  They  will  then  be  asked  to  generate  situational 
item  stems  by  writing  brief  descriptions  of  situations  they  have  witnessed  or 
experienced,  in  which  a  cadet  demonstrated  a  high  or  low  level  of  performance  on 
one  of  the  target  dimensions. 

•  Develop  skill  measure:  We  will  create  a  past  behavior  record  form  that  asks  the 
respondent  to  describe  (a)  a  situation  from  the  respondent’s  recent  past  that  required 
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the  respondent  to  exhibit  behavior  relevant  to  the  target  dimension,  (b)  the  action 
taken  by  the  respondent  in  response  to  the  situation,  and  (c)  the  outcome  or  result  of 
the  behavior.  We  will  then  conduct  two  workshops  to  collect  examples  of  past 
behavior.  We  will  ask  participants  to  describe  one  or  two  ROTC-relevant  behavioral 
examples  from  their  own  past  for  each  dimension  using  the  Past  Behavior  Record 
form.  We  will  then  develop  an  evaluation  guide  by  (a)  conducting  a  workshop  in 
which  knowledgeable  cadets  or  cadre  officers  rate  the  effectiveness  of  each 
behavioral  example,  (b)  inducing  the  principles  underlying  the  rating  process,  (c) 
developing  rating  scales  for  each  performance  dimension  that  contain  behavioral 
statements  describing  high,  medium,  and  low  levels  of  effectiveness  for  each  facet  of 
the  dimension,  and  (d)  combining  these  scales  with  rating  instructions.  The 
evaluation  guide  will  then  be  pilot  tested. 

Assemble  action  control  and  habits  measures:  We  will  create  an  action  control  scale 
for  this  study  by  writing  instructions  and  assembling  the  items  from  Diefendorff  et 
al.  (2000)  into  a  questionnaire  format.  We  will  develop  a  scale  to  assess  habitual 
tendencies  with  respect  to  relevant  performance  dimensions  by  adapting  the  Self- 
Report  Habit  Index  (Verplanken  &  Orbell,  2003)  to  our  target  behaviors. 

Prepare  for  data  collection:  We  will  identify  participating  ROTC  programs  for  data 
collection  and  obtain  IRB  approval  at  each  university. 


Goal-Driven  Perception  and  Cognition  in  Complex  Social  Environments 

Contract#:  W74V8H-05-K-0003  Contract  Dates:  1/1/05-12/31/07 

Institution:  Arizona  State  University  PI:  Douglas  T.  Kenrick 

Co-PI:  Steven  L.  Neuberg 
Liaison:  TBD 

Problem(s)/Research  Question(s)  -  How  do  common  social  goals  relevant  to  military  situations 
(e.g.,  self-protection,  other-protection,  retribution)  influence  early-stage  cognitive  processing  of 
complex  social  environments  (e.g.,  which  person  in  a  crowd  we  choose  to  pay  attention,  how  we 
interpret  this  person’s  facial  expression).  How  do  these  goal-driven  processes  shape  important 
down-stream  behavioral  decisions  (e.g.,  to  shoot  vs.  not  shoot  a  possible  enemy  combatant)? 

Technical  Barrier(s)  -  Until  recently,  little  effort  has  been  made  to  link  theory  about 
fundamental  social  goals  and  their  associated  emotions  to  theory  about  early-stage  information 
processing,  or  to  use  modem  methods  of  cognitive  science  to  rigorously  explore  the  possible 
influences  of  such  goals  on  cognitive  processing. 

Significance/Impact  for  Basic  Research  -  This  research  will  help  conceptually  integrate  two 
important,  but  disconnected,  research  literatures  while  also  providing  empirical  data  that  tests 
hypotheses  about  goal  influences  on  cognitive  processing  of  complex  social  environments. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  for 
identifying  those  individuals  most  susceptible  or  resistant  to  particular  goal-related  information 
processing  biases,  identifying  those  individuals  most  in  need  of  training  to  overcome  such  biases, 
and  designing  training  programs  to  mitigate  such  biases.  We  suggest  that  our  research  may  be 
most  directly  relevant  to  the  following  applied  programs: 

•  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force  Soldier  (ATO: 
IV.HS.  2002.01). 

•  Future  Force  Warrior  Training  (WP  215) 

•  Ground  Systems  Team  Training  (WP  302) 

•  VICTOR:  Virtual  Individual  and  Collective  Training  for  Future  Warriors  (WP  233) 

•  Assessing  and  Developing  Leaders  for  Special  Operations  Units  (WP  218) 

Overview 

Soldiers  often  encounter  extremely  complex  situations  rich  in  social  information  about 
which  they  must  rapidly  make  difficult,  life-or-death  decisions.  Cognitive  limitations  dictate, 
however,  that  one  cannot  attend  equally  to  each  individual  in  such  settings  or  to  all 
characteristics  of  any  given  individual.  Rather,  attentional  and  information  processing  must  be 
directed  selectively  to  a  small  subset  of  individuals  and  characteristics.  The  selective  direction 
of  cognitive  processing  often  occurs  automatically,  without  conscious  intent,  and  is  highly 
suscepible  to  biases  that  can  potentially  lead  to  serious  errors  (e.g.,  failing  to  process  clues  of  an 
impending  fatal  ambush  or  mistaking  an  innocent  civilian  for  an  armed  insurgent).  The  aim  of 
the  proposed  research  is  to  assess  how  common  social  goals — self-protection,  other-protection, 
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and  retribution — shape  attention  and  information  processing  in  complex  social  environments  and 
thereby  influence  important  down-stream  behavioral  decisions. 

Research  Approach 

We  plan  10  laboratory  experiments  designed  to  test  hypotheses  derived  from  our 
conceptual  framework.  Each  experiment  employs  established  research  paradigms  and 
methodologies  used  in  social  psychology  and  cognitive  science;  manipulates  participants’  self¬ 
protection,  other-protection,  and/or  retribution  goals;  assesses  individual  differences  in  the 
chronic  status  of  these  goals;  confronts  participants  with  complex  social  scenes  in  which  target 
individuals  differ  in  their  sex,  age  (combatant  age  or  not),  group  membership  (ingroup,  outgroup, 
or  outgroup  ally),  facial  expression  (angry  or  not),  and/or  possession  of  a  weapon;  and  examines 
how  these  goals  alter  visual  attention  to,  encoding  of,  memory  for,  and/or  behavioral  decisions 
about  the  different  individuals  in  the  scenes. 

Accomplishments  -  N/A  -  New  Start 

Contributions  to  Basic  Research 

This  research  will  make  several  important  contributions:  (1)  it  will  theoretically  link 
research  literatures  addressing  how  fundamental  human  goals  affect  information  processing  with 
ecologically  inspired  theories  of  motivation  and  cognition;  (2)  it  will  more  specifically  generate  a 
framework  for  understanding  how  goals  related  to  self-protection,  other-protection,  and 
retribution  shape  early-stage  attention  and  interpretation  of  complex,  information  rich  social 
environments,  and  thereby  influence  subsequent  behavioral  decision  making;  and  (3)  it  will 
provide  rigorous  empirical  tests  of  the  hypotheses  derived  from  this  framework.  In  these  ways, 
this  research  will  expand  upon  the  field’s  understanding  of  motivated  social  perception  and 
cognition. 

Potential  Army/Military  Applications 

Our  conceptual  framework  offers  specific  predictions  about  how  individuals  motivated 
by  a  range  of  social  goals  will  differ  in  how  they  process  complex  social  information  and  make 
behavioral  decisions.  It  is  thus  highly  relevant  to  understanding  how  a  soldier’s  perceptual, 
cognitive,  and  decision  making  processes  may  depend,  for  example,  on  whether  he  or  she  is 
primarily  concerned  in  the  moment  with  self-protection,  protection  of  a  friend  or  comrade, 
retribution  for  an  attack  on  one’s  country  or  the  death  a  comrade,  or  the  safety  and  well-being  of 
innocents  in  the  field.  Specifically,  this  research  should  provide  useful  information  and 
techniques  for  (1)  selecting  personnel  into  appropriate  task  roles  (e.g.,  the  research  paradigms,  or 
adaptations  of  them,  should  be  useful  for  identifying  those  individuals  most  susceptible  or 
resistant  to  particular  goal-related  biases),  (2)  identifying  those  individuals  most  in  need  of 
training  to  overcome  such  biases,  and  (3)  designing  training  programs  to  mitigate  such  biases. 

Future  Plans 

Our  plans  for  Year  1  are:  (1)  to  create  the  photo  stimuli,  goal  manipulation  films,  and 
“video  game”  required  to  run  our  proposed  experiments;  (2)  to  begin  Experiments  1  (exploring 
visual  attention  to  heterogeneous  arrays  of  target  individuals),  4  (extending  the  findings  of 
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Experiment  3),  and  7  (extending  the  findings  of  Experiment  8)  and  (3)  to  begin  and  complete 
Experiments  3  (exploring  rapid  categorical  “enemy/not-enemy”  decisions)  and  8  (testing 
hypotheses  about  how  stimuli  related  to  different  motives  interfere  with  one  another). 
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An  Integration  of  Motivation  Theories 


Contract  #:  DASW01-04-K-0001  Contract  Dates:  12/12/03-12/1 1/07 

Institution:  The  Hebrew  University  of  Jerusalem  PI:  Avraham  N.  Kluger 

Liaisons:  Trueman  Tremble,  SARU 
Robert  Solick,  LDRU 

ProbIem(s)/Research  Question(s)  -  Does  thinking  about  the  basic  challenges  for  human 
survival  show  us  ways  to  integrate  variety  of  motivation  theories? 

Technical  Barrier(s)  -Multiple  well-accepted  mid-range  theories  of  motivation  exist,  most  of 
which  are  related  to  each  other  and  inform  each  other.  However,  these  theories  use  different 
language  and  labeling  and  come  from  different  research  traditions  that  mask  useful 
commonalities. 

Significance/Impact  for  Basic  Research  -  Recognize  how  scattered  theories  are  related  to  each 
other,  which  may  lead  to  a  more  parsimonious  understanding  of  human  motivation,  and  improve 
the  communicability  of  knowledge. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Various  Army  training  programs 

•  LEADERSHIP:  Developing  leaders  in  a  changing  army  (WP  103) 

Overview 

Consider  what  strategies  humans  can  adopt  to  guarantee  survival.  Apparently,  one 
winning  strategy  is  sticking  to  what  works,  that  is,  maintenance  of  routines.  Maintaining 
routines  is  advantageous  because  any  routine,  on  average,  guarantees  survival.  One  can  infer 
that  maintaining  routines  guarantees  survival  because  performers  of  non-adaptive  routines  are 
likely  to  be  eliminated  sooner  or  later  by  nature.  Yet,  this  line  of  thought  has  one  fatal  flaw.  If 
you  only  successfully  maintain  routines,  sooner  or  later  you  will  be  eliminated  by  nature  because 
once  the  environment  changes  you  will  be  left  without  an  appropriate  behavioral  repertoire  to  fit 
the  new  environment.  So,  if  maintaining  routines  is  fatal,  what  might  be  the  alternative? 

Perhaps,  a  strategy  of  exploration  and  constant  change  may  be  more  adaptive:  investing 
efforts  in  learning,  exploring  and  experimenting  is  most  likely  to  enable  you  to  adapt  to  dynamic 
changes  in  the  environment  (food,  mates,  and  social  structures)  and  to  help  you  optimally  exploit 
the  environment  to  guarantee  survival.  Yet,  this  thinking,  too,  has  one  fatal  flaw.  A  strategy  of 
experimentation  is  patently  dangerous.  One  can  infer  that  experimentation  is  dangerous  by 
thinking  about  the  probability  that  you  will  die  either  in  an  act  of  an  experimentation  or  in  an  act 
of  repeating  a  routine.  Obviously,  a  tested  routine  is  far  safer  than  experimenting  with  the 
unknown.  Thus,  if  you  adopt  an  experimentation  strategy  you  are  likely  to  be  eliminated  by 
nature  before  too  long. 

Thus,  rather  than  by  choosing  between  maintaining-routines  and  change,  managing  a 
balance  between  these  strategies  may  enhance  fitness.  To  manage  this  preservation  versus 
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change  conflict,  two  systems  might  have  evolved.  One  system  might  have  evolved  to  maintain 
routines,  using  signals  of  pain  to  prevent  deviation  from  routine.  Another  system  might  have 
evolved  to  explore  the  advantages  of  novel  behaviors,  using  signals  of  reward  to  attract  us  to 
novel  and  successful  change.  Since  the  goals  of  these  systems  are  in  conflict  they  might  have 
evolved  as  separate  systems  that  compete  within  the  organism  for  behavior  monitoring  and 
control.  These  systems  may  provide  us  maximum  behavioral  plasticity  in  the  same  metaphorical 
manner  in  which  the  muscles  operate  smoothly  with  two  opposing  (contracting  and  relaxing) 
forces.  Indeed,  multiple  motivation  theories  seem  to  reflect  similar  concerns  and  offer  a 
complementary  picture  of  the  processes  involved  in  adaptation  (Kluger  &  Ganzach,  in  press). 

The  thoughts  about  these  two  systems  lead  to  multiple  theoretical  questions.  For  the 
purpose  of  this  document,  I  report  accomplishments  in  one  research  track  that  has  both  clear 
theoretical  resolution  and  readily  recognizable  applied  implications  for  the  Army1.  In  this  track, 
the  theoretical  question  was  “Do  different  theories  reflecting  preservation  and  change  strategies, 
yield  similar  predictions?”  To  probe  this  question,  my  group  considered  how  different  theories 
will  handle  a  practical  question:  What  motivates  people  more  -  positive  or  negative  feedback? 


Research  Approach 

To  demonstrate  the  viability  of  thinking  about  preservation  and  change  strategies,  a  two- 
stage  approach  is  taken.  The  first  stage,  the  essence  of  this  project,  was  to  develop  theory  by 
reviewing  existing  theories,  and  searching  for  commonalities  and  discrepancies.  The  second 
stage,  where  feasible,  was  to  empirically  test  specific  hypotheses  generated  within  each  of  these 
research  programs.  In  the  current  report,  the  empirical  tests  were  based  on  laboratory 
experiments. 

Accomplishments 

In  this  period,  my  group  (Levontin,  Kluger,  &  Van  Dijk,  under  review)  have  explored  the 
commonalities  and  discrepancies  between  two  leading  motivation  theories,  exposed 
discrepancies,  and  offered  a  reconcilation.  This  work  is  now  composed  as  a  theoretical  paper 
which  is  now  under  a  second  revision  for  consideration  for  publication  in  a  special  issue  on 
theory  in  the  Journal  of  Applies  Psychology  (see  brief  review  below).  Also,  data  in  initial 
support  of  the  theory  is  now  available  (see  Figures  below). 

Two  leading  motivation  theories  —  goal-orientation  theory  (Dweck,  1986)  and  self¬ 
regulation  theory  (Higgins,  1997)  -  both  appear  to  reflect  the  strategies  of  maintenance  versus 
change  and  yet  are  somewhat  inconsistent.  Dweck  (1986),  in  her  child  development  research, 
proposed  that  motivation  can  be  guided  by  orientation  toward  one  of  two  different  classes  of 
achievement  goals:  learning  goals  and  performance  goals.  Individuals  with  learning  goals  seek 
to  increase  their  competence,  to  understand  or  master  something  new.  They  view  achievement 
situations  as  opportunities  for  acquiring  new  skills,  extending  mastery  or  developing  and 


1  In  a  second  research  track  supported  by  this  grant,  I  have  shown  (Kluger,  Stephan,  Ganzach,  &  Hershkovitz,  2004) 
that  one  prediction  of  prospect  theory,  which  won  Daniel  Kahneman  the  Nobel  Prize  in  Economics  in  2002,  is 
largely  valid  for  a  maintaince  system  but  not  for  a  change  system. 
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improving  ability  (Ames  &  Archer,  1988;  Butler,  1987,  1992;  Dweck,  1986;  Dweck  &  Leggett, 
1988;  Nicholls,  1984).  The  motivation  that  results  from  learning  goals  (even  when  faced  with 
obstacles)  is  characterized  with  interest,  enjoyment,  positive  affect,  effort,  and  the  assessment  of 
performance  progress  relative  to  past  performance.  On  the  other  hand,  individuals  with 
performance  goals,  seek  to  establish  the  adequacy  of  their  ability  both  in  their  own  eyes  and  in 
the  eyes  of  others  and  to  avoid  giving  evidence  of  its  inadequacy.  They  view  achievement 
situations  as  tests  or  measures  of  competence.  The  motivation  that  results  from  performance 
goals  is  characterized  by  concerns  regarding  normative  ability,  desire  to  achieve  successful 
outcomes  or  to  avoid  unsuccessful  ones  relative  to  others,  and  an  inclination  to  prove  high  ability 
by  achieving  success  with  little  effort.  The  consequences  of  performance  goals,  when  faced  with 
obstacles,  are  helplessness,  anxiety,  negative  affect,  risk  aversion  and  low  persistence  (Ames  & 
Archer,  1988;  Butler,  1987,  1992;  Dweck,  1986;  Dweck  &  Leggett,  1988;  Nicholls,  1984). 
Thus,  it  seems  that  goal-orientation  theory  reflects  both  a  maintenance  strategy  (performance 
goal)  and  a  change  strategy  (learning  goal). 

Higgins’  (1997;  1998)  extended  the  basic  hedonic  principle  -  approaching  pleasure  and 
avoiding  pain  -  and  offered  regulatory-focus  theory  that  describes  important  differences  in  the 
processes  through  which  people  approach  pleasure  and  avoid  pain.  Specifically,  the  theory 
proposes  that  people  have  two  basic  regulatory  focus  systems.  One  system  regulates  the 
achievement  of  rewards  and  focuses  individuals  on  promotion  goals,  while  the  other  system 
regulates  the  avoidance  of  punishments  and  focuses  individuals  on  prevention  goals.  Thus,  it 
seems  that  regulatory  focus  theory  too  reflects  both  a  conservation  strategy  (prevention  focus) 
and  a  change  strategy  (promotion  focus). 

Now  consider  predictions  that  can  be  generated  by  these  two  apparently  similar  theories. 
What  motivates  people  more  -  positive  or  negative  feedback?  Self-regulation  theory  suggests 
that  under  prevention  focus  negative  feedback  motivates  more  than  positive  feedback.  That  is, 
people  that  sense  that  their  routines  are  failing  will  be  highly  vigilant  to  correct  the  recognized 
problem.  In  contrast,  goal-orientation  theory  (Dweck,  1986)  suggests  that  positive  feedback 
motivates  more  than  negative  feedback  when  people  hold  performance  goals  (maintenance 
strategy).  That  is,  people  performing  a  complex  duty  (required  to  learn  new  complex 
procedures)  can  be  debilitated  by  the  news  that  they  are  failing  or  that  they  cannot  maintain  their 
expected  organizational  roles.  To  resolve  these  conflicting  predictions  the  current  work 
considered  expectancy  theory  (Vroom,  1964)  as  a  bridge.  Expectancy  theory  suggests  that 
motivation  is  determined  by  two  antecedents:  Expectancy  to  succeed  in  the  future  (E)  and 
valence  (importance  to  oneself)  of  the  outcome  (V).  The  predictions  of  goal-orientation  may 
pertain  largely  to  feedback-sign  effects  on  belief  that  one  can  succeed  in  the  future  (E)  while  the 
predictions  of  regulatory-focus  theory  pertain  largely  to  feedback-sign  on  urgency  to  resolve  the 
problem  (V).  Hence,  our  theoretical  resolution  suggests  that: 

HI:  When  maintenance  strategy  drives  behavior  (prevention  focus  or  performance  goals) 
failure  will  both  increase  V  (drive  sensed  as  an  urgency  to  solve  the  problem)  and  reduce  E 
(belief  in  one’s  abilities). 

This  prediction  of  what  happens  when  people  receive  feedback  and  are  operating  with  a 
maintenance  strategy  is  presented  in  the  graph  below.  Following  this  graph  are  data  in  support 
of  this  model. 
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To  test  our  proposal  for  theoretical  reconciliation,  we  ran  an  experiment.  In  the 
experiment,  we  measured  participants'  (N  =  122)  levels  of  E  and  V  before  and  after  bogus 
failure  (N=122).  We  randomly  assigned  participants  who  performed  an  anagram  task  to  three 
motivational  manipulations:  (a)  Goal  orientation  (performance  goal)  was  manipulated  using  task 
instructions  (Nicholls,  1984;  Steele- Johnson  et  al.,  2000);  specifically,  participants  were 
instructed  that  performance  on  memory  tasks  reflects  basic  cognitive  capacities  and  that  the 
higher  their  underlying  cognitive  capacities  are,  the  better  their  performance  on  such  a  task 
should  be;  (b)  Regulatory  focus  (Prevention  focus)  was  manipulated  using  the  reward  for 
participating  -  participants  were  told  they  might  lose  some  money  or  not  (Shah,  Higgins  & 
Friedman,  1998).  They  were  told  that  they  are  expected  not  to  miss  more  than  10%  of  all 
possible  words.  Although  payment  for  participating  in  the  experiment  is  10  Shekels  it  is  possible 
for  them  to  lose  5.  They  would  not  lose  five  shekels  if  they  missed  10%  or  less  of  all  the  possible 
words,  but  they  would  lose  five  shekels  if  they  missed  more  than  10%  of  all  the  possible  words; 
and  (c)  Control  group  participants  did  not  get  any  information  regarding  the  task.  Results  (see 
Graph  below)  were  supportive  of  our  hypothesis.  That  is,  eliciting  either  performance  goals  or 
prevention  focus  (presumed  theoretically  similar  variables)  both  increased  valence  and  decreased 
expectancy. 

Figure  2 


Performance  goals 


Prevention  focus  Control 
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Contributions  to  Basic  Research 

The  current  work  demonstrates  that  thinking  about  strategic  challenges  for  survival  helps 
to  organize  apparently  unrelated  theories  into  one  conceptual  framework  that  facilitates  our 
understanding  of  motivation  and  reactions  to  feedback. 

Potential  Army/Military  Applications 

If  our  hypothesis  is  further  validated,  our  theory  exposes  a  serious  difficulty  in  applying 
performance  appraisal  in  the  military.  The  difficulty  stems  from  the  expectation  that  the  effect  of 
failure  on  expressed  motivation  and  actual  performance  is  likely  to  be  a  combined  effect  of 
feedback  on  urgency  (valence)  and  on  belief  (expectancy).  If  we  assume  that  in  the  military 
most  performance  evaluations  are  administered  in  an  organizational  context  in  which  avoiding 
harm  (maintenance)  is  emphasized  over  searching  for  opportunities  (change),  then  we  can 
predict  that  the  motivational  effects  of  feedback  on  the  force  on  average  are  going  to  be  nil  due 
to  the  opposite  effects  that  feedback  sign  has  on  the  two  antecedents  of  motivation  (valence  and 
expectancy).  This  call  for  a  review  of  the  goals  of  performance  appraisal  systems  in  the  military 
and  search  for  alternatives  that  appear  both  theoretically  and  empirically  more  effective. 
Whereas  the  current  work  imply  risks  for  the  Army  in  using  feedback,  the  future  plan  of  my 
groups  are  geared  towards  developing  theory-based  techniques  to  enhance  performance  via 
FEEDFORWARD.  Initial  applications  in  various  organizations  suggest  that  these  products  will 
be  highly  relevant  applied  alternatives  that  can  be  used  in  the  Army 

Future  plans 

To  continue  developing  a  theory,  I  seek  to  build  it  in  light  of  a  practical  question:  How 
one  should  use  feedback,  performance  appraisal,  etc.  in  organizations?  The  bulk  of  the  data 
suggests  that  systems  such  as  performance  appraisal  are  not  very  effective.  To  search  for 
alternatives,  the  general  theory  will  be  expanded  with  the  applied  problem  in  mind. 

Several  motivation  theories  suggest  that  humans,  in  addition  to  routine  maintenance 
versus  change  conflict,  another  set  of  motivationally  conflicting  challenges:  within  species 
competition  versus  cooperation.  This  conflict  appears  to  be  found  in  various  theories  under 
different  labels  such  as,  need  for  power  versus  need  for  affiliation;  individualism  versus 
collectivism  (e.g.,  Colquitt,  Noe,  &  Jackson,  2002);  agency  versus  communion  (e.g.,  Brunstein, 
Schultheiss,  &  Grassmann,  1998;  Fournier  &  Moskowitz,  2000)  and  dominance  versus  love 
(e.g.,  Tracey,  1994;  Tracey,  Ryan,  &  Jaschik-Herman,  2001).  Thus,  it  appears  that  this  second 
strategic  conflict  is  also  very  basic  to  human  behavior.  In  the  next  period,  I  plan  to  explore  how 
this  second  conflict  modulates  the  first  one.  This  exploration  will  be  geared  both  towards  further 
theoretical  development  and  towards  developing  new  applications  in  organizations. 

At  the  practical  level,  I  am  working  on  developing  tools  for  organizational  change 
predicated  on  the  (to  be  tested)  assumption  that  evoking  both  promotion  focus  (drive  to  change) 
coupled  with  strong  feeling  of  belongings  (camaraderie)  produce  a  mental  state  that  is  most 
amenable  for  fast  learning  and  growth.  To  create  a  sense  of  both  belongings  and  promotion,  I 
propose  that  feedback  interventions  be  preceded  by  FEED-FORWARD  interventions.  In  feed¬ 
forward,  people  (organizations)  are  coached  as  to  express  their  dreams,  ideals  and  wishes.  After 
stating  dreams,  a  plan  with  goals  and  means  is  set  to  achieve  those  dreams.  Next,  people  answer 
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the  following  question:  "Imagine  that  you  applied  your  plan,  will  it  take  you  to  your  dream?"  If 
one  discovers  a  discrepancy  between  the  imagined  future  and  one's  dreams  the  process  is 
repeated  as  to  revise  the  means,  goals,  or  even  the  dream.  When  feed-forward  yields  no 
discrepancy  the  process  is  complete.  This  mental  exercise  may  help  align  deep-seated 
motivations  with  explicit  goals;  hence,  discovering  the  shortest  route  to  the  dream  fulfillment, 
and  guaranteeing  the  creation  of  strong-goal  commitment.  Feed-forward  alone  may  provide 
positive  and  lasting  (transformational)  effects.  Such  interventions  may  work  because  they 
simultaneously  induce  a  strategy  of  change  coupled  with  a  strategy  of  cooperation  (with  the 
consultant,  organization). 
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International  Military  Education  and  Training:  A  Sociological  Analyses 

Contract  #:1435-01-01 — 31 151  Contract  Dates:  06/01/2001-1 1/30/04 

Institution:  Northwestern  University  PI:  Charles  Moskos 

Overview  and  Research  Approach 

International  Military  Education  and  Training  (IMET)  is  the  cornerstone  of  professional 
military  education  for  non-American  officers  in  the  United  States.  At  the  acme  of  IMET  are 
those  international  officers  (IOs)  who  attend  American  war,  command  and  staff  colleges  -  some 
400  annually.  Many  of  these  are  destined  to  become  senior  officers  in  their  home  militaries. 

This  report  is  based  on  personal  interviews  with  82  IOs  at  the  senior  military  schools  in 
the  United  States  in  each  of  the  services.  Interviews  were  also  conducted  with  American 
students  and  staff  members  of  these  same  schools.  Issues  covered  include  an  overview  of  the 
IMET  experience  from  the  IO  viewpoint  such  as  housing,  family,  sponsors,  friendships,  field 
trips,  classroom  atmosphere,  and  curriculum.  The  mutual  perceptions  of  IOs  and  fellow 
American  students  of  each  other  were  also  examined. 

The  overriding  finding  is  that  the  IMET  experience  is  typically  viewed  in  very  favorable 
terms.  Not  only  do  IOs  become  incorporated  into  American  professional  militaiy  education,  but 
American  students  also  gain  a  unique  military  learning  experience  by  interacting  with  IOs. 

In  sum,  IMET  is  a  key  resource  serving  long-term  American  security  and  one  that 
deserves  utmost  support  from  the  national  policy-making  community. 

Accomplishments 

The  military  senior  command  in  the  Joint  Chiefs  of  Staff  has  been  briefed  on  the  findings 
of  the  research.  Steps  are  being  introduced  to  make  medical  insurance  more  widely  available  to 
family  members,  rethinking  the  nature  of  security  classification  for  IOs,  and,  more  tentatively, 
incorporating  content  on  multinational  issues  in  the  curriculum  of  the  war  and  staff  colleges.  To 
a  major  degree,  these  initiatives  were  influenced  by  the  research  reported  herein. 

Contributions  to  Basic  Science 

The  findings  are  based  on  a  qualitative  methodology;  namely,  personal  interviews  with 
IOs  and  American  students.  The  collected  data  are  directly  relevant  to  research  on  generic 
multinational  institutions,  e.g.  multinational  political  bodies,  multinational  corporations,  non¬ 
government  organization.  In  particular  the  research  findings  bear  on  the  relative  balance 
between  cultural  differences  and  personality  variables  in  multinational  interaction  and  relations. 
In  particular,  the  research  pertains  to  Gert  H.  Hofstede’s  on  the  cultural  consequences  of 
international  differences  in  work-related  values. 

The  data  also  indicate  that  models  of  international  cooperation  must  not  rely  only  on 
econometric  analyses  and  “rational  man”  premises.  Social  psychological  and  cultural  variables 
are  central  to  any  understanding  of  international  military  cooperation.  The  eminent  psychologist 
Leon  Festinger  developed  a  “comparison  theory”  in  the  1950s.  This  is  to  say  that  humans 
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evaluate  themselves  and  their  relations  with  others  not  so  much  by  objective  standards,  but  by 
comparisons  with  people  around  them.  Similarly,  the  concept  of  “reference  groups”  greatly 
enlightens  studies  of  international  relations.  This  is  to  say  that  people  get  their  cues  from 
meaningful  groups  rather  than  a  generalized  other. 

Potential  Army/Military  Applications 

International  military  cooperation  is  a  key  element  in  military  effectiveness.  The 
significance  of  effective  coalition  warfare  can  scarcely  be  exaggerated.  This  is  especially  so 
since  the  events  of  9/1 1  and  the  subsequent  war  against  terrorism.  The  findings  have  special 
relevance  for  furthering  cooperation  of  non-American  militaries  in  mutli-national  military 
missions.  Indeed,  the  relationship  between  American  military  officers  and  those  of  other 
nationalities  can  be  a  major  factor  in  fostering  pro- Americanism  in  coalition  missions. 

At  the  more  immediate  level,  the  research  made  concrete  suggestions  on  how  to  improve 
IOs  integration  into  the  American  milieu,  both  academically  and  socially.  Certain  problem  areas 
were  analyzed  with  attendant  recommendations  for  improvement.  These  include:  (1)  making 
medical  insurance  available  to  the  family  members  of  all  IOs;  (2)  decreasing  the  material 
unavailable  to  IOs  because  of  security  classifications;  (3)  adding  curriculum  content  on 
multinational  operations  and  current  international  issues;  and  (4)  extending  more  effort  to 
incorporate  the  spouses  of  IOs  into  the  American  social  scene. 

Future  Plans 

Follow-up  research  will  be  conducted  at  multi-national  headquarters  to  ascertain  what 
steps  can  be  taken  to  improve  international  military.  This  research  will  also  enable  us  to  make 
informative  comparisons  of  American-IOs  in  other  settings.  As  always,  the  principal 
investigator  will  conduct  research  on  sociological  topics  requested  by  senior  military  officers  and 
Defense  Department  officials. 
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Comprehension  and  Memory  of  Spatial  and  Temporal  Event  Components 

Contract  #:  DASW01-02-K-0003  Contract  Dates:  05/01/02-04/30/04 

Institution:  University  of  Notre  Dame  PI:  Gabriel  A.  Radvansky 

Liaison:  Tonia  Heffner 

Problem(s)/Research  Question(s)  -  How  do  people  differ  in  the  ability  to  detect  and  process 
changes  in  spatial  and  temporal  event  information  for  on-going  situations? 

Technical  Barrier(s)  -  Models  and  theories  of  event  comprehension  and  memory  are  only 
currently  emerging. 

Significance/Impact  for  Basic  Research  -  Understanding  how  people  monitor  and  update 
changes  in  situations  will  allow  for  more  developed  models  of  how  people  comprehend  and 
remember  information  about  events  in  the  world  as  they  are  unfolding. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  FUTURE-TRAIN:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  Combat 
Team  Commanders  and  Staffs  (WP  211). 

•  MEASURE  DIGITAL:  Defining  and  Measuring  Digital  Skill  Proficiency  (WP  234). 

•  Future  Force  Warrior  Training  (WP  2 1 5). 

•  VICTOR:  Virtual  Individual  and  Collective  Training  for  Future  Warriors  (WP  233). 

•.  SELECT21:  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force 
(WP  257)  and  ATO:  IV.HS.2002.01. 

•  Perform  M21 :  Performance  Measures  for  21st  Century  Soldier  Assessment  (WP  104). 
Overview 

When  people  are  presented  with  information  about  events,  in  order  to  successfully 
comprehend  and  remember  that  information,  they  need  to  be  able  to  deal  with  the  changes  that 
occur.  Situations  and  events  are  dynamically  changing  over  time.  As  such,  people  need  to 
identify  when  these  changes  have  occurred,  and  update  their  understanding  of  the  event  in  a  way 
that  takes  these  changes  into  account.  How  do  people  keep  track  of  these  changes?  Are  some 
people  better  at  monitoring  these  changes?  What  makes  them  more  efficient? 

There  are  many  components  of  a  situation.  In  an  overview  of  research  on  event  memory 
and  comprehension,  Zwaan  and  Radvansky  (1998)  delineated  major  dimensions  that  people 
monitor,  including  space,  time,  entities,  causality,  and  intentions.  Recent  work  by  Radvansky  and 
Copeland  (2000;  2001;  2004;  Radvansky,  Copeland,  Berish  &  Dijkstra,  2003;  Radvansky, 
Copeland  &  Zwaan,  2003;  Radvansky,  Zwaan,  Curiel  &  Copeland,  2001)  has  shown  that  the 
ability  to  monitor  these  changes  is  not  related  to  general  mental  capacity  such  as  memory  span 
(i.e.,  the  amount  of  information  that  a  person  can  hold  in  mind  at  one  time).  Instead,  it  appears  to 
involve  mental  processes  and  abilities  that  are  specific  to  understanding  situations  and  events. 

•  The  focus  of  this  project  is  on  changes  in  space  and  time.  That  is,  when  there  has 
been  a  shift  in  space  (an  entity  is  in  a  new  location)  or  time  (a  significant  period 
of  time  has  passed),  people  need  to  know  what  elements  from  the  previous 
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situation  are  still  relevant  and  what  are  not.  For  example,  if  a  person  learns  that 
someone  has  traveled  from  New  York  to  London,  after  noticing  this  change,  the 
person  needs  to  mentally  keep  track  of  the  fact  that  the  traveler  had  brought  a 
computer  disk  with  classified  information  with  her,  and  that  she  has  left  behind 
her  book  of  contacts.  Although  space  and  time  are  basic  components  of  the  world, 
people  do  not  always  actively  monitor  and  update  their  understanding  of  events 
after  changes  in  space  and  time  (Zwaan  &  von  Oostendorp,  1993;  Zwaan, 
Radvansky,  Hilliard,  &  Curiel,  1998). 

•  We  assess  spatial  and  temporal  updating  using  our  knowledge  of  how  people  use 
mental  representations  called  mental  models.  Our  hypothesis  is  that  people 
proceed  through  a  series  of  stages. 

o  During  the  first  stage,  a  person  detects  a  change  in  space  or  time. 

o  During  the  second  stage,  a  person  creates  a  mental  representation  that 
takes  into  account  that  new  spatial  location  or  time  period.  The  ability  to 
engage  in  this  second  stage  depends  on  the  success  of  the  first  stage. 

o  Finally,  during  a  third  stage,  the  new  mental  model  is  created  that 
coordinates  the  relevant  information  from  the  old  situation.  Specifically, 
old  elements  that  remain  relevant  should  be  maintained  in  the  new  mental 
model.  Furthermore,  elements  that  are  no  longer  relevant  should  not  be 
included  in  the  new  model.  By  looking  at  the  success  of  people  to  track 
and  update  this  information  we  hope  to  be  able  to  identify  the 
circumstances  in  which  people  are  able  to  do  this  correctly,  and  to  identify 
ways  to  correct  any  deficiencies  in  doing  so. 

•  More  recently,  we  have  extended  our  efforts  beyond  language  comprehension  to 
interactive  experiences  with  desktop  virtual  reality  environments.  We  are  looking 
at  how  the  spatial  updating  effects  that  have .  been  observed  in  language 
comprehension  extend  to  actual  experiences  with  events.  This  line  of  work  has 
looked  at  two  issues. 

o  The  first  is  the  idea  that  when  there  are  changes  in  spatial  location, 
information  about  entities  that  are  carried  over  to  the  new  location  should 
continue  to  be  available  in  memory,  whereas  information  about  entities 
that  are  left  behind  in  an  old  location  should  become  less  available  (e.g., 
Glenberg,  Meyer,  &  Lindem,  1987). 

o  The  second  is  the  idea  that  as  a  person  moves  through  a  space,  the  further 
away  things  are  from  the  current  location,  the  less  available  information 
about  that  entity  should  be  (Morrow,  Greenspan  &  Bower,  1987).  This  is 
called  a  spatial  gradient.  By  looking  at  how  changes  in  space  during  an  on¬ 
going  event  are  disrupting  thinking.  By  knowing  this  we  can  target  those 
aspects  of  the  situation  that  are  giving  people  the  most  trouble,  allowing 
training  and  technology  to  target  these  deficits. 
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Research  Approach 

In  some  of  our  experiments,  people  were  given  a  series  of  narratives  to  read,  one  sentence 
at  a  time.  Reading  times  were  recorded.  The  length  of  reading  times  gives  an  index  of  reading 
difficulty.  Ideally,  when  an  event  shift  occurs,  people  will  exert  more  effort  to  update  their 
understanding  of  the  situation,  and  reading  times  will  increase.  Thus,  it  was  expected  that  a 
noticed  change  in  space  or  time  would  show  a  slow-down  in  reading  times. 

In  addition,  in  some  expeirments,  reading  was  interrupted  with  probe  words.  On  critical 
trials,  these  probe  words  assessed  the  ability  to  update  event  understanding.  The  task  was  to 
indicate  whether  these  words  referred  to  the  narrative.  The  expectation  was  that  people  would 
repond  faster  when  they  referred  to  an  element  that  was  part  of  the  current  situation,  and  slower 
when  it  did  not.  Our  key  interest  was  in  the  relation  between  the  various  measures,  which 
assessed  different  stages  of  event  updating.  Specifically,  people  who  were  successful  at  earlier 
stages  should  be  more  successful  at  later  stages. 

In  our  work  using  virtual  reality,  people  navigated  through  virtual  reality  environments 
with  some  goal  in  mind.  In  some  cases  the  goal  was  to  move  objects  from  room  to  room.  In 
these  cases,  we  would  look  at  the  availability  of  information  about  an  object  as  a  function  of 
whether  it  was  currently  being  carried,  or  whether  the  object  had  been  put  down  and  was  now  in 
a  different  room  from  the  person.  This  was  done  by  stopping  the  progress  in  the  environment 
and  presenting  people  memory  probes  consisting  of  object  names.  The  task  was  to  indicate 
whether  the  object  was  one  that  was  currently  being  carried,  one  that  was  just  set  down,  or  some 
other  object. 

In  other  cases,  people  memorized  the  layout  of  a  building  prior  to  virtual  reality 
navigation.  This  was  done  to  assess  both  the  influence  of  prior  knowledge,  as  well  as  spatial 
distance,  on  the  availability  of  event  knowledge.  After  memorization,  people  moved  about  in 
different  locations  within  the  virtual  reality  environment.  At  different  points,  they  made  location 
judgements  about  various  objects. 

Accomplishments 

The  results  of  our  experiments  are  consistent  with  our  hypotheses.  Specifically,  the 
degree  to  which  people  show  effects  of  situation  model  updating  at  later  stages  is  contingent  on 
their  success  at  earlier  stages.  The  data  from  one  of  our  our  experiments  looking  at  the  updating 
of  temporal  event  information  is  presented  in  the  figures  below. 


92 


10°0| 
o  ! 


Condition  Effect  of  Detection  of  New  Time  Frame 


Figure  1 .  Reading  times  (in  ms  /  syllable)  Figure  2.  Relation  between  the  effect  of  the 
for  sentences  that  either  do  or  do  not  detection  of  a  new  time  frame,  and  the  effect 

contain  a  temporal  shift.  of  response  times  to  objects  maintained 

across  temporal  boundaries. 

The  results  of  this  work  support  our  predictions.  Moreover,  the  ability  to  update  one’s 
understanding  of  the  event  in  one  area  is  related  to  the  ability  to  update  in  other  ways. 

We  also  have  some  data  from  virtual  reality  studies  looking  at  spatial  updating.  What  we 
have  observed  is  that  there  are  two  aspects  of  interaction  with  a  situation  influences  the  ability  to 
monitor  what  is  going  on.  First,  things  that  people  are  currently  associated  with  or  carrying  are 
more  available  than  things  that  have  been  just  set  down,  even  if  the  person  is  still  in  the  same 
room  with  that  object.  The  second  is  that  when  people  move  from  one  spatial  location  (room)  to 
another,  information  about  the  on-going  event  is  disrupted.  What  is  most  interesting  is  that 
knowledge  of  even  what  one  is  currently  carrying  in  the  new  spatial  location  is  disrupted  after  a 
spatial  shift.  This  is  shown  in  the  figure  below. 
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Figure  3.  Response  times  (in  ms)  to  object  name  probes  as  a  function  of  whether  people 
had  just  made  a  spatial  shift  or  not. 


93 


Our  studies  with  virtual  reality  have  also  found  that  when  people  move  from  one  room  to 
the  next,  even  in  a  well-known  area,  information  about  the  room  just  left  drops  in  availability  as 
people  mentally  work  to  understand  the  new  situation.  The  figure  below  shows  the  ability  to 
identify  information  about  objects  when  a  person  has  moved  from  one  room  ( Source )  through  a 
second  room  (Path)  and  ending  up  in  a  final  ( Goal)  room.  The  data  also  present  the  availability 
of  objects  from  some  Other  room.  People  are  slowest  for  objects  from  the  room  they  have 
recently  moved  from.  This  suggests  that  shifts  in  location  disrupt  the  ability  to  retrieve 
information  about  recent  event  components. 


Room 

Figure  4.  Response  times  to  memory  probes  (object  name  pairs)  as  a  function  of  distance 
from  the  subjects’  current  position  in  the  virtual  environment. 

Contributions  to  Basic  Research 

This  project  will  provide  us  with  a  more  advanced  understanding  of  how  people  track 
changes  in  descriptions  of  situations  that  occur  in  the  world.  This  research  follows  several  recent 
developments  in  mental  model  theories  and  extends  them  by  looking  more  precisely  and  the 
stages  that  are  involved  in  updating  understanding.  This  more  precise  analysis  will  allow  for  a 
clearer  understanding  of  how  people  comprehend,  monitor  and  remember  changes  in  events  and 
situations  in  the  world  based  on  the  information  they  receive.  From  these  data,  we  hope  to 
develop  a  means  of  identifying  when  people  are  having  trouble  updating  event  information,  and 
provide  a  more  targeted  means  of  correcting  those  problems. 

Potential  Army/Military  Applications 

The  potential  applications  of  this  research  for  the  Army  are  in  the  areas  of  training  and 
system  interfaces.  In  the  area  of  training,  this  research  will  provide  insight  into  the  specific 
difficulties  that  people  have  updating  their  understanding  of  events.  Knowing  in  what  ways 
people  have  trouble  tracking  changes  in  space  and  time  will  allow  for  the  development  of 
procedures  that  would  directly  target  such  deficits.  In  addition,  because  the  modem  soldier 
needs  to  interact  with  sophisticated  technologies  that  involve  tracking  various  entities  through 
time  and  space  (whether  they  be  enemy  forces,  one’s  own  supplies  or  friendly  forces),  knowing 
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the  capabilities  and  limits  of  human  event  comprehension  and  memory  will  help  in  the  design  of 
these  systems.  Spatial  and  temporal  information  that  is  readily  tracked  by  people  would  need 
either  minimal  or  no  feedback  from  the  system.  However,  those  aspects  of  spatial  and  temporal 
processing  that  are  more  difficult  would  be  a  higher  priority  for  that  system,  in  terms  of 
augmenting  the  soldier’s  current  capabilities. 

Future  Directions 

Currently  we  are  continuing  to  follow  two  lines  of  experimental  inquiry.  The  first  is  that 
we  are  continuing  to  explore  the  ability  of  people  to  update  spatial  and  temporal  information 
about  a  situation  that  they  acquire  from  descriptions  of  events.  More  specifically,  we  continue  to 
look  at  people’s  abilities  to  detect  spatial  and  temporal  shifts,  to  create  new  frameworks  for  when 
such  shifts  occur,  to  maintain  information  that  continues  to  be  relevant  after  an  event  shift  has 
occurred,  and  finally,  to  remove  information  from  the  current  stream  of  thought  that  is  no  longer 
relevant  because  of  the  event  shift.  The  other  line  continues  looking  at  the  ability  to  process 
spatial  shifts,  and  how  these  spatial  shifts  affect  the  ability  to  monitor  other  aspects  of  an  event. 
This  is  being  done  by  having  people  travel  through  desktop  virtual  reality  environments.  The 
aim  of  this  line  of  research  is  to  look  at  how  discoveries  in  language-based  understand  of 
situations  and  events  can  be  extended  to  more  realistic  situations.  The  desktop  virtual  reality 
environment  is  helpful  because  it  allows  us  to  construct  various  aspects  of  a  spatial  environment 
to  adequate  test  their  influence  on  thought. 
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Development  and  Empirical  Evaluation  of  a  Theory  of  Socially  Competent 
Job  Performance  for  United  States  Army  Junior  Commissioned  Officers 

Contract  #:  DASW01 -02-K-0002  Contract  Dates:  5/1/02-8/3 1/04 

Institution:  Personnel  Decisions  Research  Institutes,  Inc.  PI:  Robert  J.  Schneider 

Liaison:  Peter  J.  Legree,  RACO 

ProbIem(s)/Research  Question(s)  -  What  is  the  nature,  and  what  are  the  causes,  of  socially 
competent  work  behavior  in  junior  commissioned  Army  officers? 

Technical  Barrier(s)  -  Neither  a  well-developed  theory  of  socially  competent  work  behavior 
nor  adequate  measures  of  social  knowledge  and  motivation  are  available. 

Significance/Impact  for  Basic  Research  -  This  research  resulted  in  the  first  comprehensive 
theory  of  socially  competent  work  behavior  ever  developed  and  an  innovative  and  valid  measure 
of  social  knowledge. 

Potential  Transitions  —  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in 
the  following  ARI  work  programs  and  projects: 

•  SELECT21 :  Selection,  Classification  and  Performance  Metrics  for  the  Objective  Force 
(WP  257) 

•  Accelerating  Leader  Development  (WP  269) 

•  LEADERSHIP:  Developing  Leaders  in  a  Changing  Army  (WP  103) 

•  Perform  M2 1 :  Performance  Measures  for  2 1  st-Century  Soldier  Assessment  (WP  1 04) 

•  Assessing  and  Developing  Leaders  for  Special  Operations  Units  (WP  2 1 8) 

Overview 

Social  competence  is  critically  important  for  Army  officers.  The  purpose  of  this  research 
was  to  enhance  understanding  of  what  makes  a  junior  commissioned  officer  socially  competent. 
To  do  this,  we  built  on  previous  theoretical  work  by  Hunter  (1983),  J.  P.  Campbell  (1990),  and 
Motowidlo,  Borman,  and  Schmit  (1997)  that  focused  on  understanding  job  performance  at  a 
more  general  level.  This  earlier  work  has  two  things  in  common:  (1)  one  must  have  a  clear 
understanding  of  the  job  performance  that  is  to  be  predicted,  and  (2)  abilities,  personality  traits, 
and  other  individual-differences  characteristics  predict  job  performance  primarily  or  exclusively 
through  performance-relevant  knowledge.  Our  research  used  the  evolving  framework  developed 
in  this  earlier  work,  but  focused  specifically  on  the  interpersonal  aspects  of  job  performance, 
something  that  had  not  been  done  previously.  A  rigorously  developed  and  tested  theory  of  social 
competence  would  not  only  facilitate  selection,  classification,  and  training  of  junior 
commissioned  Army  officers,  but  would  also  add  significantly  to  the  evolving  understanding  of 
job  performance. 

Research  Approach 

We  began  our  work  by  specifying  a  general  theory  of  the  types  of  characteristics  that 
should  predict  socially  competent  job  performance.  Components  of  the  theory  that  we  believed 
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would  predict  social  job  performance  were:  interpersonal  personality  traits  (for  example, 
dominance  and  charisma),  social  intelligence,  general  intelligence,  social  knowledge,  and  social 
motivation.  Then,  we  identified  the  major  social  job  performance  dimensions.  The  next  step 
was  to  develop  measures  of  each  attribute  in  our  general  theory  of  social  competence.  Existing 
measures  were  obtained  where  possible,  but  it  was  necessary  to  develop  a  new  social  knowledge 
test  (SKT),  social  motivation  measure,  and  social  job  performance  measure. 

The  SKT  was  based  on  social  scenarios  derived  from  our  social  performance  model. 
These  were  refined  and  elaborated  in  workshops  with  subject  matter  experts  (SMEs),  and  the 
knowledge  requirements  for  each  scenario  (e.g.,  what  goals  should  one  have?  What  would  be 
effective  and  ineffective  ways  to  achieve  those  goals?)  were  fleshed  out  in  additional  workshops. 
We  used  this  information,  supplemented  by  other  information  available  on  military  websites  and 
other  relevant  source  materials,  to  create  scripts  for  each  scenario  for  videotaping.  The  SKT 
consists  of  20  such  scenarios.  Examinees  view  a  scenario  and  then  write  down  all  the  effective 
and  ineffective  behaviors  they  can  spot.  Responses  are  thus  open-ended,  as  opposed  to  multiple- 
choice  or  some  other  format  specifying  response  options  from  which  to  select.  We  also 
developed  a  measure  to  assess  people’s  motivation  to  display  each  aspect  of  social  performance 
in  our  social  performance  theory.  Finally,  we  developed  a  social  performance  measure  covering 
the  various  dimensions  that  we  found  to  constitute  socially  competent  job  performance.  The 
examinees'  social  performance  was  rated  by  those  who  interacted  with  them  often  enough  to 
know  their  social  performance  well. 

We  tested  our  theory  by  administering  instruments  measuring  its  various  components  to 
advanced  (3nl-year  and  beyond)  ROTC  cadets  and  midshipmen.  We  regarded  these  individuals 
as  good  surrogates  for  junior  commissioned  officers,  since  they  are  in  training  to  become  junior 
commissioned  officers.  We  developed  and  tested  five  separate  social  competence  theories  (all  of 
the  same  general  form)  for  each  major  dimension  of  social  job  performance  that  we  identified. 
This  was  important  because  different  things  predicted  different  aspects  of  social  performance. 
The  study  design  was  based  on  examining  correlations  between  study  variables  and  using 
advanced  statistical  techniques  to  determine  whether  the  data  corroborated  our  theoretical 
predictions. 

Accomplishments 

This  research  yielded  several  important  accomplishments.  First,  our  approach  to 
measuring  social  knowledge  turned  out  to  be  quite  viable.  We  were  able  to  get  SKT  scorers  to 
agree  in  their  evaluations  of  SKT  responses,  despite  the  fact  that  those  responses  were  open- 
ended  and,  in  some  cases,  required  significant  judgment  to  score.  More  importantly,  examinees’ 
overall  social  knowledge  scores  showed  substantial  correlations  with  three  out  of  five  social 
performance  dimensions:  Effective  Supervision,  Social  Presence,  and  Inteipersonal  Sensitivity, 
all  of  which  are  instrumental  to  success  as  a  junior  commissioned  Army  officer.  Second,  our 
results  corroborated  many  of  our  theoretical  predictions.  The  general  form  of  our  theory,  which 
incorporates  those  predictions,  is  shown  in  Figure  1 . 
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Space  limitations  preclude  a  full  description  of  our  results.  We  note,  however,  that  our 
key  hypothesis,  that  social  intelligence  would  predict  social  job  performance  only  through  its 
relationship  to  social  knowledge,  received  support  for  three  out  of  five  social  job  performance 
dimensions,  as  did  our  prediction  that  cognitive  ability  would  predict  social  job  performance 
through  its  relationship  to  social  knowledge  (though  in  one  case  it  also  predicted  social  job 
performance  independently).  These  results  make  sense  in  light  of  the  fact  that  social  intelligence 
and  general  intelligence  should  cause  socially  effective  work  performance  only  if  they  yield 
knowledge  of  what  to  do  during  social  interactions  that  play  out  in  social  scenarios. 

Contributions  to  Basic  Science 


This  project  advanced  understanding  of  interpersonal  job  performance  in  several 
important  ways.  We  identified  the  major  dimensions  of  social  job  performance,  and  formulated 
and  tested  a  family  of  theories  explaining  differences  in  those  social  job  performance 
dimensions.  Our  theoretical  predictions  received  significant  support  in  our  data. 
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Methodologically,  we  showed  that  a  scenario-based  approach  to  social  knowledge  testing 
administered  on  video  using  an  open-ended  response  format  with  right  and  wrong  answers  is 
viable.  This  type  of  measurement  is  more  realistic  than  paper-and-pencil  tests  with  response 
options  already  provided.  This  project  also  yielded  several  useful  products,  including  the  SKT 
and  the  social  performance  measure  we  developed  based  on  our  social  performance  model.  We 
also  developed  a  social  motivation  measure,  though  this  will  require  further  research  and 
development  work  before  it  can  be  used  operationally  since  it  did  not  conform  to  certain 
theoretical  predictions. 

Potential  Army/Military  Applications 

A  relatively  direct  application  of  our  research  would  be  to  take  the  social  performance 
measure  we  developed  for  this  project  and  use  it  as  the  basis  for  development  of  a  computerized 
assessment  and  development  tool.  This  could,  for  example,  be  adapted  for  use  in  the 
"Accelerating  Leader  Development"  project.  Our  social  performance  instrument  could  be  placed 
on  an  Army  website,  and  Army  leaders  could  then  receive  self-assessment-based  feedback,  or 
feedback  on  the  same  social  performance  dimensions  from  their  superiors,  peers,  and/or 
subordinates. 

The  SKT  developed  and  validated  as  part  of  this  project  could  be  used  in  a  variety  of 
ways.  For  example,  the  SKT’s  scoring  key  provides  a  wealth  of  information  about  social  tacit 
knowledge  necessary  for  effective  interpersonal  leadership  among  junior  commissioned  Army 
officers.  As  such,  our  results  would  fit  well  into  projects  such  as  "Developing  Leaders  in  a 
Changing  Army."  Among  the  stated  objectives  of  this  project  is  “establishing  an  empirical  link 
between  measures  of  social  tacit  knowledge  and  performance  outcomes.”  Data  from  our  project 
can  be  used  to  address  this  issue.  Our  results  could  also  be  used  to  develop  social  tacit 
knowledge  by  adapting  our  SKT  for  use  as  a  training  tool.  This  also  seems  consistent  with  the 
"Developing  Leaders  in  a  Changing  Army"  Project.  For  example,  the  20  video-based 
interpersonal  scenarios  in  the  SKT  could  be  placed  on  DVD.  After  responding  to  each  scenario 
(or  the  entire  SKT),  the  scenarios  could  be  repeated  for  trainees,  with  a  voiceover  that  identifies 
the  targeted  social  behaviors  and  explains  why  they  are  effective  or  ineffective. 

A  more  ambitious  training-related  application  would  be  to  pause  the  videotaped  scenarios 
in  the  SKT  at  critical  points,  require  that  the  examinees  assume  the  role  of  one  or  more 
characters  in  the  scenarios,  and  have  them  say,  in  "real  time,"  what  they  believe  the  characters 
should  say  at  those  junctures  to  be  effective.  A  camera  and  microphone  could  be  placed  on  the 
examinees  at  the  computer  stations  where  they  would  be  taking  the  test  to  create  an  audiovisual 
record  of  their  responses.  We  could  then  create  sample  audiovisual  responses  spanning  the  range 
of  effectiveness  levels  and  create  audiovisual  behaviorally  anchored  rating  scales  (AVBARS), 
whereby  different  videotaped  performances  would  be  calibrated  to  different  effectiveness  levels. 
These  AVBARS  could  be  used  as  a  self-development  tool  by  showing  officers  their  performance 
and  comparing  it  to  different  levels  of  performance  based  on  the  AVBARS.  If  used  for  formal 
training,  trainers  could  administer  the  AVBARS  to  trainees  and  score  their  audiovisual  responses 
to  assess  their  training  needs  and  plan  training  interventions.  Learning  points  for  training 
programs  could  be  derived  from  differences  in  various  AVBARS  levels. 

These  results  could  also  be  extended  to  selection  and  training  of  Army  recruiters.  This  is 
a  job  that  requires  significant  interpersonal  knowledge,  skill,  and  ability,  since  recruiters’  ability 
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to  appeal  to  potential  recruits  and  sell  the  Army  lifestyle  determines  their  success.  Some  of  our 
predictors  may  add  value  to  selection  systems  for  these  recruiter  jobs.  In  addition,  the 
methodology  we  used  to  develop  SKT  scenarios  could  be  applied  to  develop  recruiter-specific 
scenarios,  with  associated  scoring  guidelines  that  target  effective  and  ineffective  recruiter 
behaviors  within  those  scenarios.  This  would  be  relevant  to  projects  such  as  "RECRUITER: 
Improved  Screening  Tools  for  Recruiters  and  Station  Commanders." 

Final  Summary 

In  this  project,  we  formulated  and  tested  a  theory  of  socially  competent  job  performance 
for  junior  commissioned  Army  officers.  We  also  developed  an  innovative  measure  of  social 
knowledge,  and  demonstrated  the  viability  of  that  approach  to  social  knowledge  testing.  Aside 
from  demonstrating  the  validity  of  the  SKT,  our  most  striking  result  was  that,  for  three  out  of 
five  social  performance  dimensions,  social  intelligence  influenced  socially  competent  job 
performance  only  through  its  relationship  with  social  knowledge,  just  as  our  theory  predicted. 
Our  results  indicate  that  interpersonal  personality  traits,  social  intelligence  and  social  knowledge 
all  predict  important  aspects  of  junior  commissioned  Army  officer  job  performance.  They  also 
establish  that  general  intelligence  is  predictive  of  several  aspects  of  interpersonal  job 
performance. 

The  primary  implication  of  this  research  for  the  Army  is  that  measures  of  social 
intelligence  and  certain  interpersonal  personality  traits  should  be  incorporated  into  selection  and 
classification  systems  for  junior  commissioned  Army  officers.  In  addition,  since  social 
knowledge  will  be  more  predictive  of  social  performance  than  social  intelligence,  it  should  be 
considered  seriously  for  selection/classification  and  training  applications. 

Having  demonstrated  the  importance  of  social  knowledge  to  junior  commissioned  officer 
job  performance  and  the  viability  of  our  social  knowledge  measurement  approach,  we 
recommend  that  our  SKT  be  adapted  for  use  for  other  Army  ranks,  and  for  specific  occupational 
specialties  involving  significant  interpersonal  interactions.  For  example,  with  some  adaptation, 
the  SKT  could  be  generalized  to  NCOs.  It  could  also  be  adapted  for  use  with  higher-ranking 
officers,  where  understanding  of  more  sophisticated  organizational  politics  becomes  more 
important.  Similarly,  new  scenarios  could  be  developed  for  occupations  such  as  recruiters, 
medical  and  counseling  personnel,  and  special  operations  unit  leaders,  among  other  possibilities. 

Future  research  in  this  area  should  seek  to  develop  measures  of  social  skills  (using 
instrumentation  that  does  not  rely  on  self-report)  and  evaluate  whether  the  type  of  social 
knowledge  measured  by  our  SKT  influences  social  performance  only  through  social  skills.  In 
addition,  social  self-regulation  should  be  incorporated  into  our  social  competence  theory.  For 
example,  how  are  social  goals  maintained  in  the  face  of  initial  failures  to  translate  them  into 
effective  socially  performance? 
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Social  Structures  Affecting  Army  Performance 

Contract  #:  DASW  0100  K  0016  Contract  Dates:  8/17/00-12/31/04 

Institution:  The  University  of  Maryland  PI:  David  R.  Segal 

Co-PI:  Mady  Wechsler  Segal 

Problem(s)/Research  Questions  -  What  are  the  social  structures,  social  processes  and  social 
psychological  characteristics  (e.g.,  attitudes,  values)  that  lead  young  Americans  to  enter  the 
armed  forces  as  enlisted  personnel  or  through  officer  accession  programs? 

Technical  Barriers  -  Our  knowledge  of  the  structure  of  youth  attitudes  relevant  the  military 
service  is  not  well  developed,  and  while  these  attitudes  have  been  shown  to  change  significantly 
over  time,  the  change  processes  are  not  well  understood. 

Signiflcance/Impact  for  Basic  Research  -  Identification  of  attitude  organization  and  change 
processes  among  young  adults  will  help  us  better  understand  the  dynamics  of  this  early  stage  in 
the  life  course  and  how  they  relate  to  subsequent  behavior  in  adult  life. 

Potential  Transitions  -  Knowledge  gained  in  this  research  may  be  usefully  employed  in: 

•  Our  Army’s  recruitment  processes 

•  Our  Army’s  officer  accession  processes 

•  Our  Army’s  understanding  of  the  needs  of  soldiers  and  their  families 

•  Our  Army’s  ability  to  manage  an  increasingly  diverse  military  force 

Overview 

In  a  democracy,  the  armed  forces  reflect  to  a  considerable  degree  the  society  they  defend, 
and  members  of  the  armed  forces  participate  in  other  social  institutions  as  well.  The  intersection 
of  the  military  and  its  host  society,  including  other  social  institutions  such  as  the  family,  the  labor 
market,  and  the  educational  system,  is  an  important  component  of  both  democratic  civil-military 
relations,  and  of  military  effectiveness.  How  do  changing  attitudes  among  young  Americans 
toward  military  service,  work  in  general,  and  family  formation  and  roles  affect  our  ability  to 
recruit  and  field  an  effective  military  force?  Have  trends  in  expectations  among  young 
Americans  regarding  whether  they  will  serve  in  the  armed  forces  affected  our  ability  to  recruit? 
What  role  do  patriotic  versus  materialistic  motivations  to  serve  play  in  the  kinds  of  soldiers  we 
recruit?  How  do  educational  incentives  figure  into  these  motivations?  Such  incentives  have  been 
conceptualized  alternatively  as  compensation  in  kind  to  reward  patriotic  service,  and  as  direct 
economic  incentives.  Do  young  Americans  who  elect  to  serve  in  the  armed  forces  seek  different 
characteristics  from  their  work  than  their  peers  who  enter  the  civilian  labor  force  or  go  to  college 
after  secondary  school?  To  what  extent  do  they  feel  that  their  desires  to  start  families  are 
compatible  with  military  service?  Since  a  larger  proportion  of  the  current  young  adult  generation 
in  America  aspires  to  collegiate  education  than  ever  before,  how  does  military  service  fit  into  the 
life  plans  of  this  generation?  We  seek  answers  to  questions  such  as  these  to  better  understand  the 
linkages  between  the  armed  forces  and  society  in  America.  We  build  on  theories  that  view  the 
military  in  America  as  a  part  of  rather  than  apart  from  society. 
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Research  Approach 

Much  or  our  research  involves  the  collection  and  analysis  of  survey  data  on  a  range  of 
populations,  civilian  and  military.  Our  current  efforts  draw  upon:  an  ongoing  program  of 
research  on  the  attitudes  of  high  school  seniors  (the  Monitoring  the  Future  program)  that  has 
involved  surveys  of  every  high  school  graduating  class  since  1975  with  follow-up  surveys  of 
sub-samples  for  fourteen  years  after  graduation;  a  recent  survey  of  the  entering  class  at  a  federal 
military  academy  that  included  questions  asked  more  broadly  of  young  civilians  participating  in 
the  Monitoring  the  Future  survey;  and  a  recent  survey  of  first-term  American  soldiers  serving  in 
infantry  units  that  sought  to  measure  their  recollections  of  their  expectations  for  service  while 
still  in  high  school,  the  factors  that  motivated  them  to  enlist,  and  their  sense  of  identification  with 
military  roles  compared  to  other  social  roles  they  played,  such  as  parents  and  church  members. 

Accomplishments 

Our  research  has  contributed  to  our  understanding  of  why  some  high  school  graduates 
elect  to  enter  the  military  rather  than  pursue  alternative  post-secondary  school  activities  such  as 
college  or  civilian  employment.  We  have  documented  a  downward  trend  in  the  expectations  and 
desires  of  high  school  seniors  to  serve  in  the  armed  forces,  making  the  military  more  dependent 
on  young  men  and  women  who  had  other  plans.  Most  recently,  we  have  explored  the  structure 
of  various  motivational  factors  that  bring  people  into  the  military,  and  how  these  related  to 
whether  young  men  and  women  had  expected  to  join  the  armed  forces.  Earlier  research  had 
suggested  two  general  sets  of  motives:  those  rooted  in  patriotism  or  a  desire  for  adventure,  and 
those  rooted  in  economic  need.  Our  analysis  suggested  a  more  complex  structure.  Data  from 
our  first-term  infantrymen  confirmed  that  the  most  important  set  of  motivators  reflected  a  desire 
to  serve,  patriotism,  a  desire  for  adventure  or  challenge,  and  a  desire  to  be  a  soldier.  We  call  this 
our  institutional  factor.  We  also  confirmed  a  job-oriented  set  of  motivators,  including  a  need  to 
support  one’s  family,  a  situation  of  economic  crisis,  and  the  absence  of  other  or  better  job 
options.  We  call  this  our  occupational  factor.  However,  more  powerful  than  the  occupational 
factor  was  a  set  of  motivators  reflecting  plans  for  the  future,  including  a  desire  for  a  military 
career  (which  portends  long-term  retention),  and  a  desire  to  get  money  for  college  (which 
portends  separation  from  service  once  educational  benefits  are  earned).  We  also  identified  a 
factor  based  on  monetary  issues  independent  of  the  occupational  motivations  described  above. 
These  monetary  issues  included  joining  because  of  an  enlistment  bonus,  and  needing  money  to 
repay  college  loans. 

Only  the  institutional  factor  and  the  plans  for  the  future  factor  were  significantly  related 
to  whether  soldiers  had  anticipated  joining  the  military  while  they  were  still  in  high  school. 
Those  with  such  anticipation  we  call  high  propensity  soldiers,  while  those  without  such  plans  we 
call  low  propensity  soldiers.  As  shown  in  Figure  1,  the  high  propensity  soldiers  report  being 
strongly  motivated  by  patriotic/adventure  seeking  factors,  and  by  military  service  being 
compatible  with  their  long-term  plans,  and  placed  little  importance  on  the  need  for  a  job  or  for 
money.  For  soldiers  who  had  not  anticipated  serving,  by  contrast,  motivations  rooted  in 
patriotism,  adventure,  and  future  plans  were  unimportant.  They  placed  greater  importance  on 
motivators  reflecting  the  need  or  desire  for  a  job  or  money.  As  expectations  for  military  service 
continue  to  decline  among  high  school  seniors,  the  military  will  become  increasingly  dependent 
on  low  propensity  soldiers.  It  remains  to  be  seen  how  Army  performance  and  effectiveness  is 
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affected  by  having  increasing  proportions  of  soldiers  serving  because  they  need  the  work  or  need 
the  money. 


Figure  1.  Enlistment  Motivation  by 
Propensity  to  Serve 


In  our  research  on  the  job  attitudes  and  values  of  American  youth,  we  confirmed  the 
existence  of  six  clusters  of  job  characteristics  that  had  been  reflected  in  earlier  research 
(extrinsic;  intrinsic;  influence;  social  altruism;  security;  leisure).  We  identified  four  groups  of 
high  school  seniors  in  the  Class  of  2002  based  upon  their  plans  for  the  future  (planning  both 
college  and  military  service;  planning  college;  planning  military  service;  planning  neither),  and 
compared  them  on  these  desired  job  characteristics.  We  included  in  these  comparisons  data  from 
plebes  (freshmen)  at  a  federal  military  academy.  In  general,  the  plebes  ranked  lower  than  the 
high  school  seniors  on  the  importance  of  extrinsic  characteristics  (financial  and  social 
recognition);  no  different  on  the  importance  of  intrinsic  characteristics  (interesting  work);  higher 
on  the  importance  of  influence  characteristics  (responsibility  and  decision-making);  slightly 
higher  on  social  altruism  characteristics  (level  of  personal  contact  and  service  to  others);  lower 
on  the  importance  of  security  characteristics  (geographic  and  personal  stability);  and  lower  on 
the  importance  of  leisure  job  characteristics  (level  of  free  time  and  supervision,  job  pace). 
Exemplary  comparisons  across  all  groups  on  the  influence  dimension  and  the  security  dimension 
are  presented  in  Figure  2.  What  our  findings  suggest  overall  is  that  young  men  and  women 
entering  the  service  academy  are  not  dramatically  different  from  their  high  school  peers,  but  that 
the  differences  that  do  emerge  suggest  a  generally  good  fit  between  the  characteristics  of  military 
life  and  the  people  who  aspire  to  it.  Officer  candidates  are  less  motivated  by  a  desire  for  status, 
prestige,  and  money;  more  motivated  by  a  desire  for  responsibility;  more  motivated  to  be  of 
service  to  others;  and  less  motivated  by  desires  to  be  able  to  stay  in  one  place,  be  free  of 
supervision,  and  have  a  great  deal  of  free  time  than  their  high  school  peers. 
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Figure  2.  Influence  and  Security  Job  Values  Scales 
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Contributions  to  Basic  Science 

This  project  contributes  to  our  understanding  of  the  attitudes  and  behaviors  of 
contemporary  American  youth  at  the  important  life-course  transition  point  between  adolescence 
and  adulthood,  the  ways  in  which  work-related  attitudes  are  organized,  and  the  implications  of 
these  attitudes  for  behavior,  including  service  in  the  armed  forces.  In  part,  we  have  confirmed, 
though  replication,  theories  about  the  ways  in  which  work-related  attitudes  are  organized.  We 
have  also,  through  the  analysis  of  patterns  of  propensity  to  serve  and  actual  enlistment, 
contributed  to  understanding  of  the  relationship  between  attitudes  and  behaviors,  and  the  ways  in 
which  this  relationship  varies  among  demographic  groups.  At  the  same  time,  we  have  tested 
hypotheses  drawn  from  existing  theories  about  the  ways  in  which  motivations  to  serve  in  the 
military  are  organized,  and  while  we  have  confirmed  a  basic  dichotomy  between  patriotic  and 
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materialistic  orientations,  we  have  also  shown  that  the  actual  pattern  of  motivations  is  more 
complex  than  that. 

Our  research  on  soldiers  and  their  families  has  helped  us  to  understand  the  range  of 
tensions  that  exist  at  the  interface  of  work  life  and  family  life,  and  the  role  that  family  life  plays 
in  work  performance,  satisfaction,  and  retention.  Our  research  on  diversity  increases  our 
understanding  of  how  an  organization  that  has  traditionally  drawn  on  the  dominant  majority 
sector  of  American  society  can  adapt  to  the  changing  demographics  of  the  American  labor  force. 

Potential  Army/Military  Applications 

Our  research  program  on  the  attitudes  and  behavior  of  American  youth  is  relevant  for 
understanding  the  labor  market  environment  within  which  the  military  recruiting  process  takes 
place.  The  U.S.  Army  Recruiting  Command  has  used  the  results  of  this  research  for  several 
years,  it  was  recently  cited  in  a  National  Research  Council  study  on  the  youth  population 
conducted  for  the  Office  of  the  Secretary  of  Defense,  and  during  the  last  year  it  was  briefed  both 
to  the  Reserve  Forces  Policy  Board  and  to  the  Chief  of  Naval  Operations’  Executive  Advisory 
Panel.  Our  research  on  Army  families  has  contributed  to  the  development  of  ARI  handbooks  for 
family  support  groups  and  for  Army  leaders  who  must  provide  support  for  families  during 
deployments,  and  has  been  briefed  to  personnel  from  the  Army  personnel  chiefs  office.  Our 
research  on  the  ways  in  which  attitudes  and  behaviors  vary  by  gender,  racial,  and  ethnic  groups 
has  been  drawn  upon  as  the  Army  deals  with  issues  in  the  management  of  an  increasingly  diverse 
force. 

Final  Summary 

We  have  analyzed,  both  in  the  United  States  and  other  nations,  the  social  institutions, 
attitudes,  and  values  that  influence  people  to  select  post-secondary  school  alternatives,  and  for 
those  who  select  military  service,  the  processes  that  influence  retention  and  departure.  We  found 
expectations  of  military  service  to  be  declining  in  the  youth  population,  with  variations  by 
gender,  racial,  and  ethnic  group.  We  also  found  these  expectations  to  be  related  to  motivations 
to  serve,  with  those  who  made  later  decisions  to  serve  being  more  materialistic.  We  have  found 
that  perceptions  of  compatibility  between  military  service  and  family  life  have  an  impact  on  both 
recruitment  and  retention.  These  findings  should  feed  into  ARI  research  on  soldier  attitudes  and 
opinions,  the  management  of  attrition,  and  strategies  to  retain  quality  soldiers.  Our  research  on 
the  young  Americans  who  begin  study  at  a  federal  service  academy  shows  that  on  most 
dimensions,  they  are  similar  to  their  civilian  peers.  At  the  same  time,  there  is  a  realistic 
understanding  of  the  realities  of  military  life  that  make  service  different  from  most  forms  of 
civilian  employment.  Thus,  we  find  a  fit  between  the  individuals  who  seek  a  commission,  and 
the  conditions  of  commissioned  life.  Increasing  the  reality  of  expectations  and  understandings  of 
officer  candidate  should  support  ongoing  ARI  research  on  officer  accession  and  retention. 
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Construct  Validating  Aspects  of  the  Theory  of  Successful  Intelligence  Via  a 
Test  Battery  for  Measuring  Mental  Flexibility 

Contract  #:  DASWO-03-K-0001  Contract  Dates:  3-01-03  to  2-28-06 

Institution:  Yale  University  PI:  Robert  J.  Sternberg 

Co-PI:  Jens  F.  Beckmann 
Liaison:  Joe  Psotka,  LDRU 
Peter  Legree,  RACO 

Problem(s)/Research  Question(s)  -  Mental  flexibility — the  ability  to  cope  with  novelty  and  to 
establish  automatized  levels  of  information  processing — is  an  important  aspect  of  intelligence  as 
it  is  conceptualized  in  Sternberg’s  triarchic  theory  of  successful  intelligence.  The  purpose  of  this 
project  is  to  create  a  new  assessment  instmment  that  simultaneously  provides  partial  construct 
validation  of  the  theory  of  successful  intelligence  (Sternberg,  1985)  and  a  means  for  assessing 
mental  flexibility. 

Technical  Barrier(s)  -  There  currently  exists  no  broadly  conceived  test  of  mental  flexibility, 
hence,  a  measure  is  needed  that  will  do  justice  to  the  construct. 

Significance/Impact  for  Basic  Research  -  Given  the  importance  of  mental  flexibility  in  a 
rapidly  changing  world,  and  the  fact  that  such  flexibility  is  not  currently  assessed  within  the 
framework  of  conventional  psychometric  tests,  such  a  test  seems  to  have  practical  utility  as  well 
as  theoretical  justification.  In  battlefield  situations,  circumstances  change  rapidly,  and 
commanders  need  to  be  able  to  change  their  plans  quickly  and  effectively  in  response  to 
changing  conditions.  For  example,  the  shape  of  the  conflict  in  Iraq  changes  continually,  and 
effective  response  requires  flexibility  in  recognizing  the  changing  posture  of  the  enemy. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  SELECT21:  Selection,  Classification  and  Performances  Metrics  for  the  Future  Force  (WP 
257)  and  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force  Soldier 
(ATO:  IV.HS.  2002.01) 

•  LEADERSHIP:  Developing  Leaders  in  a  Changing  Army  (WP  103) 

•  Accelerating  Leader  Development  (WP  269)  and  ATO  ID.HS.2004.01 

•  Future  Force  Warrior  Training  (WP  215) 

•  FUTURE-TRAIN:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  Combat 
Team  Commanders  and  Staffs  (WP  211) 

Overview 

Intelligence  tests  have  always  been  subject  to  heavy  criticism.  One  of  the  most  prominent 
criticisms  regards  the  lack  of  precision  in  predicting  success  in  dealing  with  real-life  problems, 
such  as  educational  or  occupation.  Thus,  it  can  be  questioned  whether  scores  on  traditional 
intelligence  tests,  taken  alone,  are  optimal  for  predicting  success  in  problem  solving  in  a  rapidly 
changing  world  or  even  whether  the  traditional  concept  of  “intelligence”  is  wholly  adequate  for 
this  purpose.  An  appropriate  assessment  procedure  should  reflect  a  person’s  ability  to  adjust  in  a 
flexible  manner  to  changing  situational  requirements. 
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The  theory  of  successful  intelligence  specifies  not  only  the  kinds  of  broad  abilities 
(analytical,  creative,  and  practical)  that  play  a  role  in  achieving  success,  but  also  the  cognitive 
processes  required  to  apply  these  abilities  and  the  problem-solving  strategies  through  which 
success  may  be  achieved.  Within  the  framework  of  the  triarchic  theory  of  successful  intelligence, 
intelligence  is  conceptualized  on  three  different  levels.  On  the  first  level,  the  focus  is  primarily 
on  the  interaction  of  cognitive  components  as  the  elementary  and  universal  units  of  information 
processing  (componential  subtheory).  The  second  level  deals  with  the  relationship  of  intelligence 
to  the  experience  of  the  individual  (experiential  subtheory).  On  the  third  level,  intelligence  is 
defined  in  terms  of  the  context  in  which  intelligent  behavior  occurs  (contextual  subtheory). 

The  goal  of  this  project  is  to  test  the  experiential  subtheory,  which  specifies  that  mental 
flexibility — the  ability  to  cope  with  novelty  and  to  establish  automatized  levels  of  information¬ 
processing — plays  a  critical  role  in  the  successful  and  purposive  application  of  information¬ 
processing  components  to  adapting  to,  selecting,  and  shaping  the  environment  in  novel  situations 
(Sternberg,  1999).  Therefore,  a  multifaceted  set  of  psychometric  tests  of  flexible  thinking  that 
measures  how  well  one  can  apply  the  components  of  intelligence  in  relatively  novel  tasks  and 
situations  has  been  developed  and  is  currently  being  piloted. 


Research  Approach 

The  theory  of  successful  intelligence  differs  somewhat  from  conventional  theories  of 
intelligence  in  its  conceptualization  of  what  mental  flexibility  is  and  where  it  belongs  in  a  theory 
of  intelligence.  Within  this  framework,  mental  flexibility  serves  as  a  link  between  creative 
intelligence  and  the  experiential  subtheory.  Creative  intelligence  allows  the  individual  to  apply 
information-processing  components  to  generate  novel  and  interesting  ideas  or  to  build  on  novel 
concepts.  Both  aspects  of  creative  intelligence  require  mental  flexibility  as  an  antecedent  for 
demonstrating  creativity.  In  other  words,  mental  flexibility  is  a  necessary  but  not  sufficient 
condition  for  convergent  and  divergent  thinking.  Performance  on  a  test  of  mental  flexibility 
should  also  provide  incremental  prediction  of  success  beyond  that  obtained  with  conventional,  g- 
based  tests.  Thus  far  our  assessment  battery  of  mental  flexibility  contains  two  major  sub- 
categories  of  assessment:  Flexible  Inference  and  Flexible  Mapping.  Let  us  look  at  each  sub¬ 
category  of  assessment. 

1.  Flexible  Inference:  Figure  1  gives  an  example  for  an  item  using  shapes  for  a 
classification  task.  Here,  the  participant  must  select  the  pair  of  answer  choices  that  constitutes  the 
best  match  to  the  target  on  the  left  side,  based  on  their  common  properties. 
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Figure  1.  Example  of  a  classification  task  created  for  the  test  for  mental  flexibility-Flexible 
Inference  (part  1  of  the  given  item  triplet). 

The  correct  answer  to  this  item  would  be  the  upper  left  pair,  referring  to  the  overall  shape 
that  the  elements  of  this  pair  have  in  common  with  the  target  on  the  left.  Although  this  paradigm 
is  typically  used  to  assess  fluid  intelligence,  we  believe  it  can  be  adapted  to  test  mental  flexibility. 
The  participant  will  be  confronted  next  with  the  same  target  (on  the  left  side  of  the  screen)  and 
even  with  the  same  set  of  stimuli  on  the  right  side,  but  they  are  now  rearranged  (see  Figure  2). 
Because  the  rule  of  inference  used  for  the  previous  problem  (star-shape)  is  no  longer  valid,  the 
participant’s  focus  must  switch  to  other  characteristics.  Now,  the  consideration  of  the  number  of 
attributes  (dots,  spikes,  and  rays)  will  lead  to  the  correct  answer  in  this  item  (lower  left  pair).  The 
shapes  are  then  presented  a  third  time  in  yet  another  arrangement,  and  the  participant  must  once 
again  infer  the  relationship  that  links  them  together.  All  items  in  the  Flexible  Inference  test  are 
thus  arranged  in  item  triplets.  The  stimuli  are  shapes  (as  in  this  example),  numbers,  or  words. 


Figure  2.  Example  of  a  classification  task  created  for  the  test  for  mental  flexibility  -  Flexible 
Inference  (part  2  of  the  given  item  triplet). 

To  be  successful  on  these  items,  flexible  use  of  different  frames  of  reference  for  familiar  stimuli 
is  necessary.  The  ability  to  inhibit  experience  gained  on  previous  items  is  the  prerequisite  for  utilizing 
different  cognitive  approaches  to  the  same  set  of  stimuli.  This  paradigm  is  reminiscent  of  research  done 
by  Gestalt  psychologists.  For  example,  the  water  jug  studies  conducted  by  Luchins  (1942)  showed  that 
people  may  become  biased  by  experience  to  prefer  certain  approaches  to  a  problem,  which  may  then 
prevent  them  from  identifying  the  most  efficient  strategies  to  solve  subsequent  problems. 
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Our  approach  is  based  on  the  assumption  that  we  can  utilize  the  performance  differences 
between  two  different  item  classes  we  have  combined  in  the  test.  In  the  Flexible  Inference  test,  items 
that  ask  for  the  inference  of  domain-typical  classification  rules  (e.g.,  focusing  on  numerical 
characteristics  in  numbers)  represent  one  class.  Items  that  require  the  inference  of  classification  rules 
based  on  domain-atypical  characteristics  of  the  stimuli  (e.g.,  number  of  vowels  in  words)  represent  the 
other  item  class.  Every  item  triplet  consists  of  items  from  each  class. 

In  the  case  of  the  classification  tasks  (Flexible  Inference),  we  can  assume  that  it  is  harder 
to  find  domain-atypical  than  domain-typical  classification  rules.  We  also  expect  that  it  will  be 
more  difficult  to  identify  rules  when  an  item  (as  a  part  of  an  item  triplet)  is  preceded  by  another 
item  using  the  same  target  and  set  of  stimuli  in  which  domain-typical  characteristics  were 
relevant  for  its  solution.  In  other  words,  the  unfamiliarity  or  novelty  effect  (domain-atypical 
characteristics)  will  be  complemented  by  a  transition  effect  caused  by  the  inhibition  costs  for 
previous  perspectives  on  the  same  set  of  stimuli.  Many  of  us  have  faced  similar  problems  in 
learning  a  second  language.  The  first  language  is  a  source  both  of  positive  and  negative  transfer: 
the  more  different  the  second  language,  the  lesser  the  amount  of  positive  transfer  and  the  greater 
the  amount  of  negative  transfer.  When  soldiers  learn  to  use  new  weapons,  they  must  be  careful  to 
control  for  any  negative  transfer  so  that  they  do  not  use  the  new  weapons  improperly,  possibly 
risking  self-injury  as  a  result. 

Generally,  it  is  expected  that  the  intraindividual  variability  in  performance  scores  within  each 
item  triplet  will  be  indicative  of  people’s  ability  to  use  their  cognitive  resources  flexibly. 

2.  Flexible  Mapping:  A  similar  approach  is  used  for  another  subtest  within  the  test 
battery  for  mental  flexibility.  With  the  utilization  of  analogy  problems,  we  seek  to  gain 
information  about  a  person’s  ability  to  apply  a  previously  inferred  rule  across  different  situations. 

As  in  the  Flexible  Inference  test,  all  items  are  organized  in  item  triplets.  The  first  part  of  a  given 
item  triplet  represents  a  traditional  analogy:  A  relation  between  the  elements  of  the  analogy  stem 
must  be  inferred,  and  a  rule  based  on  this  relation  must  be  applied  to  complete  the  analogy.  In 
traditional  analogies,  the  rule  must  be  mapped  to  other  elements  from  the  same  domain  (see 
Figure  3a).  In  our  novel  tasks,  however,  we  try  to  broaden  the  mapping  distance  by  introducing  a 
domain  switch  within  each  of  the  analogy  item  triplets.  For  instance,  the  relation  between  two 
numbers  (e.g.,  88  and  22)  must  be  inferred  (the  latter  is  a  fourth  of  the  former)  and  mapped  onto 
another  domain  so  that  the  same  relation  between  two  words  (see  Figure  3b),  or  two  shapes  (see 
Figure  3c),  will  complete  the  analogy  correctly. 

If  the  rule  is  inferred  correctly,  the  participant  will  recognize  that  the  second  term  in  the 
analogy  stem  is  one-fourth  of  the  first  term.  Mapping  the  rule  into  the  verbal  domain  will  lead 
the  participant  to  choose  “GRANDMA”  because  a  grandmother  is  one  of  four  grandparents  (see 
Figure  3b).  Mapping  into  the  shape  domain  (as  required  in  the  third  part  of  the  item  triplet) 
should  result  in  choosing  the  third  answer  option.  Here  the  single  solid  triangle  represents  one- 
fourth  of  the  shape  given  (see  Figure  3  c). 
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Figure  3.  Example  of  an  analogy  task  created  for  the  test  for  mental  flexibility-Flexible  Mapping 
(a-c:  parts  1  to  3  of  the  item  triplet). 

The  purpose  of  this  procedure  is  to  have  an  indicator  of  a  person’s  ability  to  bridge 
different  mapping  distances.  Whereas  the  new  classification  tasks  (Flexible  Inference)  focus  on 
the  ability  to  infer  different  relations  flexibly,  the  focus  in  the  analogy  test  (Flexible  Mapping)  is 
on  the  ability  to  map  rules  flexibly. 

Similar  to  Flexible  Inference  (see  classification  problems),  the  item  pool  of  this  new 
assessment  procedure  also  consists  of  two  different  classes  of  items.  In  Flexible  Mapping  we 
have  “non-mapping  items,”  where  no  domain  switch  is  required  within  the  given  analogy,  and 
“mapping  items,”  where  the  domain  of  the  analogy  stem  is  different  from  that  of  the  application 
field.  Both  classes  of  items  are  represented  in  each  item  triplet.  Because  of  the  wider  mapping 
distance  to  be  bridged  in  mapping  items,  they  are  expected  to  be  harder  (mapping  costs).  A 
person’s  variability  in  performance  (within  each  item  triplet)  will  be  an  indicator  for  the 
disturbance  the  item-class  shift  causes  individually.  Our  general  expectation  is  that  the  specific 
procedure  we  have  deployed  in  both  tests,  Flexible  Mapping  and  Flexible  Inference,  will  cause 
interindividual  differences  in  levels  of  intraindividual  variability  in  performance  on  items  from 
different  classes. 

For  validation  purposes,  we  need  to  map  out  the  correlation  pattern  to  indicators  of 
related  but  not  identical  constructs  such  as  creativity,  divergent  thinking  and  fluid  intelligence. 
Therefore  we  chose  a  set  of  reference  tests  to  administer  together  with  the  tests  of  mental 
flexibility.  To  assess  divergent  thinking  abilities  we  selected  six  subtests  from  the  Berlin 
Structure  of  Intelligence  Test  (BIS,  Jager,  SuB  &  Beauducel,  1997).  Two  subtests  referring  to  the 
numerical  domain,  two  subtests  referring  to  the  figural  domain,  and  two  subtests  referring  to 
verbal  domain  represent  the  marker  reference  tests  for  divergent  thinking.  As  a  reference  test  for 
classification  (see  Flexible  Inference)  in  the  figural  domain,  we  used  a  figural  reasoning  subtest 
from  the  BIS.  As  a  second  reference  for  classification  and  adaptive  flexibility  in  the  figural 
domain  we  used  a  subtest  from  Kit  of  Factor-Referenced  Cognitive  Tests  (F-Kit,  Ekstrom, 
French  &  Harman,  1976).  We  also  use  various  subtests  from  the  F-Kit  to  measure  classification 
in  the  verbal  domain  as  well  as  for  numerical  reasoning  ability. 

To  check  whether  the  performance  scores  in  the  new  mental  flexibility  tests  are 
correlated  or  even  confounded  with  personality  characteristics  (e.g.,  openness  to  experience, 
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extraversion)  to  a  greater  extent  than  are  traditional  measures  of  intellectual  abilities,  we 
administered  the  NEO-Five  Factor  Inventory  (Costa  &  McCrae,  1985). 

Information  about  high  school  GPA  and  first-year  college  GPA  serves  as  an  external 
criterion  for  the  ability  to  deal  flexibly  with  the  adaptation  process  to  a  new  learning 
environment. 

Accomplishments 

For  Flexible  Inference  we  have  created  an  initial  item  pool  of  135  items.  Because  each 
item  cannot  be  administered  to  every  participant,  the  whole  item  pool  was  divided  into  three 
subsets.  Each  participant  was  given  45  items,  organized  into  15  item  triplets.  For  each  domain 
(numerical,  figural,  verbal)  there  are  five  item  triplets  for  every  participant.  For  Flexible 
Mapping  we  also  created  an  initial  item  pool  of  135  items.  The  same  segmentation  procedure 
performed  for  Flexible  Inference  was  applied  to  this  item  pool,  resulting  in  three  item-pool 
subsets.  All  items  are  presented  via  a  computer  program. 

As  a  first  step,  the  original  item  pools  of  both  tests  were  screened  by  experts  and  rated  for 
the  items’  domain  typicality  vs.  domain  atypicality  in  the  Flexible  Inference  test.  In  a  pre-pilot 
study,  the  tests  were  administered  to  25  subjects  to  test  the  reliability  of  the  computer  program, 
the  comprehensibility  of  the  instructions  and  the  screen  design.  Based  on  these  data  and  the 
feedback  we  received,  the  instruction  phase  was  improved.  After  the  revision  process,  we  began 
collecting  data  from  college  undergraduates. 

We  have  collected  data  from  147  (98  female,  49  male)  college  undergraduates  (18  <  age 
<  25,  mean  age:  20.3).  During  the  third  quarter  of  2004  we  recruited  subjects  from  Yale 
University.  During  the  fourth  quarter  we  are  collecting  data  from  undergraduates  from 
Quinnipiac  University. 

Contributions  to  Basic  Research 

According  to  the  theoretical  assumptions  within  the  framework  of  the  theory  of 
successful  intelligence,  mental  flexibility  reflects  the  ability  to  deal  with  novelty  and  to  establish 
automatized  levels  of  information  processing.  To  assess  this  ability  we  utilized  an  assessment 
approach  that  belongs  to  the  category  of  dynamic  testing.  As  has  been  shown  in  other  areas  (e.g., 
for  the  assessment  of  learning  ability,  see  Guthke  &  Beckmann,  2000,  2003;  see  also  Sternberg 
&  Grigorenko,  2002),  this  diagnostic  approach  represents  a  more  appropriate  way  to  assess 
intellectual  abilities  such  as  mental  flexibility.  In  contrast  to  traditional  approaches,  the  focus 
here  is  on  a  person’s  ability  to  deal  with  standardized  variations  of  test  conditions.  In  addition  to 
this  methodological  improvement,  the  study  will  provide  further  insight  into  the  cognitive 
processes  underlying  intelligent  behavior  while  adapting  to,  shaping,  and  selecting  an 
environment  in  novel  situations.  Furthermore,  we  may  gain  another  piece  of  evidence  for  the 
validity  of  the  theory  of  successful  intelligence  with  its  broader  and  therefore  better  applicable 
conceptualization  of  intelligence. 
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Potential  Army/Military  Applications 

With  the  increasing  diversity  of  missions  Army  personnel  face,  mental  flexibility 
becomes  a  more  prominent  need  in  Army  leadership.  Indeed,  one  could  argue  that  mental 
flexibility  is  the  single  most  important  attribute  commanders  and  subordinates  need  in  order  to 
succeed  in  today’s  Army.  Therefore,  appropriate  selection  tools  are  required.  By  gaining  insight 
into  the  underlying  processes  of  mentally  flexible  behavior  effective  training  tools  can  be 
developed. 

Future  Plans 

The  first  goal  of  the  upcoming  data  analyses  will  be  to  test  the  psychometric  qualities  of 
the  items  and  to  review  and  reduce  the  item  pool  as  necessary.  We  are  particularly  interested  in 
internal  validity  as  established  by  item  and  factor  analyses. 

A  second  goal  is  to  establish  the  external  validity  of  the  new  assessment  procedures.  We 
expect  different  levels  of  novelty  of  the  items  to  be  reflected  in  different  correlation  patterns  to 
reference  tests.  Another  goal  will  focus  on  the  incremental  validity  of  the  mental  flexibility  tests 
in  the  prediction  of  success  in  dealing  flexibly  with  the  adaptation  process  to  a  new  learning 
environment  while  transitioning  from  high  school  to  college. 

Our  future  recruitment  efforts  will  focus  on  the  necessity  to  obtain  representative  samples 
to  cover  a  wide  range  of  abilities.  To  fulfill  the  claim  of  having  a  multifaceted  test  battery  for  the 
assessment  of  mental  flexibility,  further  attempts  will  focus  on  the  development  of  other  tests, 
such  as  counterfactual  analogies,  where  analogies  with  premises  must  be  solved.  The  premises 
should  introduce  different  levels  of  novelty  to  the  analogy,  which  should  trigger  additional 
mental-flexibility-related  variance  in  the  performance  data.  Furthermore,  we  will  include  insight 
problems  (see  Sternberg  &  Davidson,  1995)  as  reference  tests  in  the  study  design. 
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Temporal  Investigations  into  the  Relationship  Between  Affect  and 
Discretionary  Work  Behavior 

Contract  #:  W74V8H-04-K-0001  Contract  Dates:  05/01/2004  -  04/30/2006 

Institution:  Purdue  University  PI:  Howard  M.  Weiss 

Co-PI:  Reeshad  S.  Dalai 
Liaisons:  Michelle  Zyblut,  LDRU 
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Problem(s)/Research  Question(s)  -  What  is  the  nature  of  the  structure  of  citizenship  and 
counterproductive  behavior  at  work,  collectively  known  as  discretionary  behavior?  How  does 
this  behavior  vary  over  time  within  individuals  and  how  does  the  expression  of  such  behavior 
vary  with  changes  in  persons’  affective  (emotional)  states?  Also,  what  are  variables  that  alter  the 
relationship  between  emotions  and  discretionary  behavior  at  work? 

Technical  Barrier(s)  -  Models  of  how  discretionary  behavior  changes  within  individuals  and 
the  effects  of  emotions  on  discretionary  behavior  have  not  been  well  developed.  This  gap  in  the 
literature  existed  in  large  part  because  the  methodological  and  statistical  tools  to  study  such 
phenomena  have  not  existed  until  recently. 

Significance/Impact  for  Basic  Research  -  Understanding  the  structure  of  discretionary 
behavior,  and  the  emotion-behavior  linkages,  will  lead  to  more  comprehensive  theory  and 
models  of  human  performance. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  SELECT21:  Selection,  Classification  and  Performance  Metrics  for  the  Future  Force  (WP 
257);  and  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force  Soldier 
(ATO:  IV.HS.2002.01) 

•  Accelerating  Leader  Development  (WP  269) 

•  Ground  Systems  Team  Training  (WP  302) 

•  Future  Force  Warrior  Training  (WP  215) 

Overview 

In  spite  of  the  recent  interest  in  citizenship  and  counterproductive  behavior,  as  well  as 
their  documented  importance  to  organizational  functioning,  much  remains  to  be  discovered  about 
these  criteria.  Our  program  of  research  will  incorporate  the  burgeoning  literature  on  the 
immediate  consequences  of  momentary  affective  states  (see  Brief  &  Weiss,  2002,  for  a  review  of 
this  literature)  with  the  literature  on  discretionary  work  behavior.  We  will  conduct  a  set  of 
studies  that  explore  and  map  individuals’  naturally  occurring  on-the-job  citizenship  and 
counterproductive  work  behavior,  their  emotional  antecedents,  and  variables  that  potentially 
influence  the  emotion-behavior  relationship.  In  order  to  better  understand  the  emotional  causes 
of  citizenship  and  counterproductive  behavior,  we  first  have  to  understand  the  underlying 
dimensional  structures  of  this  behavior. 

There  are  four  main  goals  of  our  project.  First,  we  propose  to  examine  the  structure  of 
work  behavior  by  determining  the  co-occurrence  of  citizenship  and  counterproductive  behaviors 
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as  well  as  when  people  switch  between  these  behaviors  over  time.  The  main  aims  of  this  study 
are  to  determine  which  behaviors  occur  together  on  a  given  time  occasion  (i.e.,  behavioral  co¬ 
occurrence)  and  predicting  changes  from  one  behavioral  state  on  one  occasion  to  the  next 
behavioral  state  on  subsequent  occasion  (i.e.,  behavioral  switching).  So  far,  the  literature  has  not 
examined  the  possibility  that  the  dynamic  structure  (i.e.,  using  “time”  as  the  level  of  analysis)  of 
citizenship  and  counterproductive  behavior  may  not  mirror  the  static  structure  (i.e.,  using 
“person”  as  the  level  of  analysis),  an  oversight  that  needs  to  be  rectified.  The  structure  of 
discretionary  behavior  also  needs  to  be  established  before  we  explore  its  emotional  antecedents. 

The  second  study  focuses  on  establishing  employees’  immediate  emotional  states  as  the 
causes  of  citizenship  and  counterproductive  work  behavior.  Research  examining  the  impact  of 
emotional  states  on  citizenship  behavior  and,  particularly,  counterproductive  behavior  is  rather 
meager.  Even  less  understood  are  the  work  events  that  instigate  the  immediate  emotional  states 
and  thereby  drive  the  display  of  this  behavior.  Although  basic  research  on  emotions  suggests 
that  momentary  affective  states  have  an  important  influence  on  both  citizenship  behavior  and 
counterproductive  behavior,  an  overarching  framework  for  studying  these  relationships  in  detail 
is  needed.  We  will  use  Affective  Events  Theory  (Weiss  &  Cropanzano,  1996)  as  a  unifying 
framework  for  understanding  these  relationships  in  detail.  A  fundamental  proposition  of  our 
research  is  that  changes  in  the  likelihood  of  engaging  in  citizenship  and  counterproductive 
behavior  over  time  are  related  to  changes  in  a  person’s  affective  states.  Therefore,  in  order  to 
test  our  predictions,  we  will  examine  how  behavior  and  emotions  change  within  a  person  rather 
than  between  people.  This  approach,  known  as  a  within-person  design,  will  allow  us  to  study 
how  key  events  in  the  workplace  and  the  way  supervisors  and  organizations  create  and  manage 
these  events  influence  employees’  behavior  and  affective  states. 

The  experience  of  negative  affective  states  does  not  always  lead  to  dysfunctional  work 
behavior;  similarly,  citizenship  behavior  does  not  always  follow  positive  affective  states.  People’s 
ability  to  “influence  which  emotions  they  have,  when  they  have  them,  and  how  they  experience  and 
express  those  emotions”  should  influence  whether  or  not  a  given  behavior  is  exhibited  (Gross,  1 999, 
p.  557).  This  process,  known  as  emotional  regulation,  must  be  integrated  into  the  study  of  the  way 
emotional  states  instigate  counterproductive  behavior.  Individuals  vary  in  their  ability  to  regulate 
their  emotions,  but  situational  constraints  may  also  determine  whether  individuals  have  the  ability  to 
regulate  and  control  their  emotions.  It  is  likely  that  the  ability  to  regulate  one’s  emotions  (or  lack 
thereof)  will  influence  the  expression  of  counterproductive  and  citizenship  behavior.  By 
investigating  the  role  of  emotional  regulation,  we  will  be  able  to  develop  a  taxonomy  of  stable 
regulatory  strategies  that  can  be  used  to  control  the  dysfunctional  consequences  of  emotional  states. 

Employees’  emotions  and  behavior  occur  in  a  work  context  that  can  play  an  important 
role  in  facilitating  or  inhibiting  certain  behavior  following  an  emotion-instigating  event. 
Therefore,  the  fourth  and  final  study  will  examine  leaders’  ability  to  regulate  their  subordinates’ 
behavior,  their  adeptness  at  identifying  others’  emotions,  and  their  skill  at  managing  and  altering 
these  emotions  in  an  effort  to  avert  counterproductive  behavior  on  the  part  of  the  employee. 

Research  Approach 

Contrary  to  traditional  cross-sectional  studies  that  look  at  differences  in  key  variables 
between  individuals,  we  are  adopting  a  within-person  approach  that  examines  how  variables 
change  within  an  individual  to  address  our  research  questions.  Given  that  the  structure  of 
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discretionary  behavior  and  the  associations  between  affect  and  such  behavior  are  constantly 
changing  and  temporally  sensitive,  we  will  be  using  a  dynamic  data  collection  tool  known  as 
Ecological  Momentary  Assessment  (EMA).  Ecological  momentary  assessment  has,  of  late, 
begun  to  gain  popularity  in  the  organizational  psychology  literature  (Weiss,  Nicholas,  &  Daus, 
1999;  Alliger  &  Williams,  1993)  because  it  allows  for  the  quantitative  analysis  of  work  behavior 
in  its  natural  context  (Hormuth,  1986)  -  not  only  the  natural  physical  context  but  also  its  natural 
temporal  context.  EMA  can  therefore  be  used  to  examine  issues  concerning  the  relatedness  of 
variables  within  persons  over  time. 

In  order  to  capture  events  as  they  occur,  we  will  be  asking  participants  to  complete 
multiple  daily  questionnaires  on  Personal  Digital  Assistants  (PDAs)  such  as  Palm  Pilots™  for 
data  collection.  Because  people  cannot  be  surveyed  constantly,  several  different  EMA  sampling 
strategies  have  been  developed.  Participants  can  be  surveyed  upon  the  occurrence  of  specified 
events,  or  after  a  fixed  or  random  amount  of  time  has  elapsed  since  the  previous  survey. 
Depending  on  the  particular  study,  the  duration  of  the  study  will  vary;  however,  surveys  will 
typically  last  10-15  working  days.  Individuals  will  be  asked  to  provide  daily  reports  about  their 
citizenship  behavior,  counterproductive  behavior,  and  affective  states. 

Accomplishments 

We  have  developed  our  first  two  studies  examining  the  structure  of  citizenship  and 
counterproductive  behavior,  and  the  relationship  between  affective  states  and  this  behavior 
within  persons  over  time.  We  have  obtained  approval  from  Purdue  University’s  Institutional 
Review  Board  (IRB)  to  proceed  with  data  collection.  In  addition,  we  are  in  the  process  of 
negotiating  with  a  Fortune  500  corporation  to  secure  a  site  for  data  collection  with  employees. 


Contributions  to  Basic  Science 

Contrary  to  most  traditional  models  of  work  performance,  our  focus  on  employees’ 
affective  states  as  a  predictor  of  performance  will  further  the  existing  literature  by  providing  a 
framework  for  understanding  how  emotions  influence  work  behavior  over  time.  In  addition,  our 
projects  will  integrate  theories  of  emotional  regulation,  affective  experiences,  and  discretionary 
behavior.  In  addition  to  the  theoretical  contribution,  the  results  of  our  projects  will  provide  the 
basis  for  developing  interventions  for  preventing  dysfunctional  work  behavior  through  the  use  of 
emotional  regulatory  strategies.  Employees  may  be  trained  to  recognize  the  interplay  between 
their  emotions  and  performance  at  work,  and  supervisors  and  organizations  can  develop  work 
environments  that  would  foster  affective  states  that  lead  to  citizenship  behavior  and  discourage 
counterproductive  behavior  through  the  reduction  of  negative  affective  states. 

Potential  Army/Military  Applications 

Negligent  and  willful  counterproductive/deviant  behavior  are  especially  of  great  importance  in 
the  military,  where  the  cost  of  such  behavior  during  combat  operations  can  often  be  measured  in 
terms  of  lost  lives:  those  of  civilians,  of  one’s  fellow  soldiers,  and — in  cases  of  excessive  brutality 
and  wanton  disregard  for  norms  of  combat — even  of  enemy  combatants.  Consequently,  it  is  no  great 
surprise  that  the  United  States  military  has  long  recognized  the  importance  of  studying 
counterproductive  (also  called  deviant  or  delinquent)  behavior  among  its  own  soldiers  (e.g.,  Bell  & 
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Holz,  1975;  Lennon,  1994).  Our  research  is  clearly  relevant  to  the  U.S.  Army  in  its  quest  for  the 
“good  soldier.”  Clear  implications  for  training  interventions  and  further  applied  research  in  Army 
settings  can  be  drawn  from  this  research. 

The  major  implication  of  the  proposed  research  is  that  behavior  and  emotions  are  transient  in  nature. 
They  are  therefore  likely  to  have  a  dynamic,  temporal  structure  that  may  not  be  identical  to  their 
static  structure.  We  suggest  first  determining  the  temporal  structure  of  discretionary  behavior  in 
civilian  settings  with  their  inherent  lower-intensity  stressors,  and  then  conducting  more  applied 
research  with  the  military  to  examine  the  extent  to  which  previous  findings  hold  up  in  more  extreme 
situations — e.g.,  patterns  of  soldier  behavior  under  enemy  fire  on  the  battlefield.  This  subsequent 
applied  military  research  can  take  the  form  not  only  of  analysis  of  emotion-behavior  and  behavior- 
behavior  links  in  simulated  real-time  battlefield  environments,  but  also  of  retrospective 
investigations  into  egregious  incidents  (e.g.,  soldiers  deliberately  shooting  their  own  colleagues, 
executing  civilians,  committing  mutiny  against  a  superior  officer,  torturing  prisoners,  etc.) 
committed  by  soldiers.  Such  research  would  provide  valuable  “red  flags”  that,  if  heeded,  could  head 
off  extremely  serious  incidents  before  they  actually  occur. 

Another  implication  of  our  proposed  research  is  that  emotional  responses  to  stressful  and/or 
aversive  events  can  be  regulated,  preventing  expression  of  counterproductive  behavior.  Emotional 
regulation  strategies  can  be  executed  either  by  the  soldier  himself  or  herself,  or  by  the  commanding 
officer.  Research  on  emotion-behavior  links  and  emotional  regulation  are  perhaps  even  more 
important  in  the  military  than  in  other  settings,  given  research  suggesting  that  verbal  behavior  in  the 
military  has  ambiguous  emotional  nuances,  making  correct  interpretations  of  the  speaker’s  intent 
especially  difficult  (Wirshbo,  1990).  Officers  unable  to  correctly  discern  their  subordinates' 
emotional  states  are  unlikely  to  be  able  to  respond  with  appropriate  emotional  regulation  strategics. 
Future  research  in  military  settings  should  therefore  examine  situational  and  individual  differences  in 
emotional  judgment  related  to  oneself  or  others. 

Additionally,  the  proposed  research  can  lay  the  groundwork  for  the  study  of  officer-sponsored 
“climates”  that  create  contingencies  between  certain  behavior  and  certain  outcomes,  thereby 
attenuating  emotion-behavior  linkages.  For  instance,  an  officer  who  clearly  articulates  and 
consistently  enforces  a  fair  and  non-discriminatory  policy  of  sanctions  for  poor  disciplinary  practices 
may  be  able  to  reduce  the  incidence  of  such  practices  regardless  of  the  frequency  and  magnitude  of 
stressors  and  consequent  aversive  reactions  experienced  by  his  or  her  subordinates.  The  findings  of 
the  proposed  studies  are  likely  to  provide  material  relevant  to  courses  in  leader  development  and 
training. 

Future  Plans 

Over  the  next  year,  we  will  complete  data  collection  for  the  behavioral  co-occurrence  and 
switching  study,  as  well  as  for  studies  2  and  3  which  examine  the  relationship  between  emotional 
states  and  discretionary  work  behavior,  and  emotional  regulation  as  a  moderator.  In  addition,  we 
will  finalize  the  design  of  study  4  which  will  examine  the  role  of  leadership  on  the  expression  of 
emotions  and  discretionary  work  behavior. 
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